Data management platform, intelligent defect analysis system, intelligent defect analysis method, computer-program product, and method for defect analysis

ABSTRACT

A data management platform for intelligently managing data is provided. The data management platform includes an ETL module configured to extract, cleanse, transform, or load data; a data lake configured to store a first group of data formed by extracting raw data from a plurality of data sources by the ETL module; a data warehouse configured to store a second group of data formed by cleansing and standardizing on the first group of data; a general data layer configured to store a third group of data formed by subjecting the second group of data to data fusion; and a data mart configured to store a fourth group of data formed by transforming the third group of data by the ETL module. The general data layer is a distributed data storage storing information available for querying. The data mart is a database of NoSQL type storing information available for computational processing.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation-in-part of U.S. application Ser. No.17/438,752, filed Dec. 3, 2020, which is a national stage applicationunder 35 U.S.C. § 371 of International Application No.PCT/CN2020/133681, filed Dec. 3, 2020. Each of the forgoing applicationsis herein incorporated by reference in its entirety for all purposes.

TECHNICAL FIELD

The present invention relates to semiconductor electronics technology,more particularly, to a data management platform for use insemiconductor electronics manufacturing, as well as an intelligentdefect analysis system, an intelligent defect analysis method, acomputer-program product, and a method for defect analysis based on thedata management platform.

BACKGROUND

The manufacturing of semiconductor electronics and devices such asdisplay panels involves a highly complex and integrated process,involving numerous processes, technologies, and equipment.

SUMMARY

In one aspect, the present disclosure provides a data managementplatform for intelligently managing data, comprising an ETL moduleconfigured to extract, cleanse, transform, or load data; a data lakeconfigured to store a first group of data formed by extracting raw datafrom a plurality of data sources by the ETL module, the first group ofdata having same contents as the raw data; a data warehouse configuredto store a second group of data formed by cleansing and standardizing onthe first group of data by the ETL module; a general data layerconfigured to store a third group of data formed by subjecting thesecond group of data to data fusion by the ETL module, wherein thegeneral data layer is a distributed data storage storing informationavailable for querying; and a data mart configured to store a fourthgroup of data formed by transforming the third group of data by the ETLmodule, wherein the data mart is a database of NoSQL type storinginformation available for computational processing.

Optionally, the fourth group of data comprising a plurality ofsub-tables having index relationship in which data are categorized basedon types and/or rules, the fourth group of data available for queryingor computational processing.

Optionally, the first group of data stored in the data lake isdynamically updated; and the first group of data comprises a real-timeupdated data stored in a Kudu™-based database, or a periodically updateddata stored in a Hadoop distributed file system.

Optionally, at least a portion of the raw data in the plurality of datasources is in a binary large object format; at least a portion of thefirst group of data extracted onto and stored in the data lake is in acompressed hexadecimal format; and at least a portion of the secondgroup of data extracted onto and stored in the data warehouse isobtained by decompressing and processing extracted data.

Optionally, the ETL module is further configured to generate adynamically updated table that is automatically updated periodically;and the general data layer is configured to store the dynamicallyupdated table comprising information on defects of interest.

Optionally, the data mart is configured to store a plurality ofsub-tables split from a respective one of data tables stored in thegeneral data layer; wherein the plurality of sub-tables comprises afirst sub-table comprising a plurality of environmental factorscorresponding to user-defined analysis criteria in an interactive tasksub-interface in communication with the data management platform; asecond sub-table comprising product serial numbers; and a thirdsub-table comprising values in the third group of data that correspondto at least one of the product serial numbers.

Optionally, the general data layer is configured to performing datafusion on at least two of parameter feature information, biographicalinformation of a manufacturing process, and defect informationassociated therewith.

In another aspect, the present disclosure provides an intelligent defectanalysis system, comprising a distributed computing system comprisingone or more networked computers configured to execute in parallel toperform at least one common task; and one or more computer readablestorage mediums storing instructions that, when executed by thedistributed computing system, cause the distributed computing system toexecute software modules; wherein the software modules comprise a datamanagement platform configured to intelligently extract, transform, orload raw data from a plurality of data sources into a managed data,wherein the raw data and the managed data comprise defect information,and the managed data is stored in a distributed manner; an analyzerconfigured to perform defect analysis upon receiving a task request, theanalyzer comprising a plurality of algorithm servers configured toobtain the managed data from the data management platform and performalgorithm analysis on the managed data to derive a result data onunderlying reasons for defects; and a data visualization and interactioninterface configured to generate the task requests and display theresult data.

Optionally, the data management platform comprises an ETL moduleconfigured to extract, transform, or load data from the plurality ofdata sources onto a data mart that is a database of NoSQL type; and uponreceiving an assigned task, a respective one of the plurality ofalgorithm servers is configured to obtain a first data from the datamart.

Optionally, the ETL module is further configured to extract, transform,or load data from the plurality of data sources onto a general datalayer that is a distributed data storage storing information; uponperforming defect analysis, the respective one of the plurality ofalgorithm servers is configured to transmit a second data to the generaldata layer; the ETL module is configured to generate a dynamicallyupdated table that is automatically updated periodically; and thegeneral data layer is configured to store the dynamically updated table.

Optionally, the software modules further comprise a query engineconnected to the data management platform and configured to obtain themanaged data from the data management platform.

Optionally, the data visualization and interaction interface isconfigured to generate a task request; the analyzer further comprises aplurality of business servers; upon receiving the task request, the oneor more of the plurality of business servers are configured to transmita query task request to the query engine; the query engine, uponreceiving the query task request from the one or more of the pluralityof business servers, is configured to query the dynamically updatedtable to obtain information on defects of interest, and transmit theinformation on defects of interest to one or more of the plurality ofbusiness servers; upon receiving defect analysis tasks, the one or moreof the plurality of algorithm servers are configured to obtain the firstdata from the data mart to perform defect analysis; and upon completionof the defect analysis, the one or more of the plurality of algorithmservers are configured to transmit results of the defect analysis to thegeneral data layer.

Optionally, the task request is an automatically recurring task request,the automatically recurring task request defining a recurring period forwhich the defect analysis is to be performed; the query engine isconfigured to query the dynamically updated table to obtain theinformation on defects of interest limited to the recurring period; andupon receiving the information on defects of interest during therecurring period, the one or more of the plurality of business serversare configured to generate the defect analysis tasks based on theinformation on defects of interest during the recurring period.

Optionally, the data visualization and interaction interface comprisesan automatic task sub-interface allowing input of the recurring periodfor which the defect analysis is to be performed.

Optionally, the task request is an interactive task request; the datavisualization and interaction interface is configured to receive auser-defined analysis criteria, and configured to generate theinteractive task request based on the user-defined analysis criteria;upon receiving the information on defects of interest, the one or moreof the plurality of business servers are configured to transmit theinformation to the data visualization and interaction interface; thedata visualization and interaction interface is configured to displaythe information on defects of interest and a plurality of environmentalfactors associated with the defects of interest, and configured toreceive a user-defined selection of one or more environmental factorsfrom the plurality of environmental factors, and transmit theuser-defined selection to the one or more of the plurality of businessservers; and the one or more of the plurality of business servers areconfigured to generate the defect analysis tasks based on theinformation and the user-defined selection.

Optionally, the data visualization and interaction interface comprisesan interactive task sub-interface allowing input of the user-definedanalysis criteria comprising the user-defined selection of one or moreenvironmental factors.

Optionally, the analyzer further comprises a plurality of businessservers; and the software modules further comprise a load balancerconnected to the analyzer, the load balancer configured to receive taskrequests and configured to assign the task requests to one or more ofthe plurality of business servers to achieve load balance among theplurality of business servers, and configured to assign tasks from theplurality of business servers to one or more of the plurality ofalgorithm servers to achieve load balance among the plurality ofalgorithm servers.

Optionally, the task requests are assigned to each of the one or more ofthe plurality of business servers based on a number of tasks currentlyperformed by each of the plurality of business servers and an amount ofcomputational burden required by each task currently performed by eachof the plurality of business servers, to achieve an optimal load balanceamong the plurality of business servers.

Optionally, the data visualization and interaction interface isconfigured to generate a task request; the load balancer is configuredto receive the task request and configured to assign the task request toone or more of the plurality of business servers to achieve load balanceamong the plurality of business servers; the one or more of theplurality of business servers are configured to transmit a query taskrequest to the query engine; the query engine, upon receiving the querytask request from the one or more of the plurality of business servers,is configured to query the dynamically updated table to obtaininformation on defects of interest, and transmit the information ondefects of interest to one or more of the plurality of business servers;the one or more of the plurality of business servers are configured totransmit defect analysis tasks to the load balancer for assigning thedefect analysis tasks to the one or more of the plurality of algorithmservers to achieve load balance among the plurality of algorithmservers; upon receiving the defect analysis tasks, the one or more ofthe plurality of algorithm servers are configured to query the firstdata from the data mart to perform defect analysis; and upon completionof the defect analysis, the one or more of the plurality of algorithmservers are configured to transmit results of the defect analysis to thegeneral data layer.

Optionally, the analyzer further comprises a cache server connected tothe plurality of business servers and the query engine; and the cacheserver is configured to store a portion of results of previouslyperformed defect analysis tasks in a cache.

Optionally, the data visualization and interaction interface comprises adefect visualization sub-interface; the defect visualizationsub-interface is configured to receive a user-defined selection of adefect to be analyzed and generate a call request; the load balancer isconfigured to receive the call request and configured to assign the callrequest to one or more of the plurality of business servers to achieveload balance among the plurality of business servers; the one or more ofthe plurality of business servers are configured to transmit the callrequest to the cache server; and the cache server is configured todetermine whether information on the defect to be analyzed is stored inthe cache.

Optionally, upon a determination that the information on the defect tobe analyzed is stored in the cache, the one or more of the plurality ofbusiness servers are configured to transmit the information on thedefect to be analyzed to the defect visualization sub-interface fordisplaying.

Optionally, upon a determination that the information on the defect tobe analyzed is not stored in the cache, the one or more of the pluralityof business servers are configured to transmit a query task request tothe query engine; the query engine, upon receiving the query taskrequest from the one or more of the plurality of business servers, isconfigured to query the dynamically updated table to obtain informationon the defect to be analyzed, and transmit the information on the defectto be analyzed to the cache; the cache is configured to store theinformation on the defect to be analyzed; and the one or more of theplurality of business servers are configured to transmit the informationon the defect to be analyzed to the defect visualization sub-interfacefor displaying.

Optionally, the portion of results of previously performed defectanalysis tasks comprises results of previously performed defect analysistasks based on automatically recurring task requests; and/or results ofpreviously performed defect analysis tasks obtained based on the querytask request.

Optionally, the load balancer is a load balancer based on Nginx™technology.

Optionally, the data mart is based on Apache Hbase™ technology; and thegeneral data layer is based on Apache Hive™ technology.

Optionally, the query engine is a query engine based on Impala™technology.

In another aspect, the present disclosure provides an intelligent defectanalysis method performed by a distributed computing system comprisingone or more networked computers configured to execute in parallel toperform at least one common task; the method comprising executing a datamanagement platform configured to intelligently extract, transform, orload raw data from a plurality of data sources into a managed data,wherein the raw data and the managed data comprise defect information,and the managed data is stored in a distributed manner; executing ananalyzer configured to perform defect analysis upon receiving a taskrequest, the analyzer comprising a plurality of algorithm serversconfigured to obtain the managed data from the data management platformand perform algorithm analysis on the managed data to derive a resultdata on underlying reasons for defects; and executing a datavisualization and interaction interface configured to generate the taskrequests and display the result data.

In another aspect, the present disclosure provides a computer-programproduct, for intelligent defect analysis, comprising a non-transitorytangible computer-readable medium having computer-readable instructionsthereon, the computer-readable instructions being executable by aprocessor, in a distributed computing system comprising one or morenetworked computers configured to execute in parallel to perform atleast one common task, to cause the processor to perform executing adata management platform configured to intelligently extract, transform,or load raw data from a plurality of data sources into a managed data,wherein the raw data and the managed data comprise defect information,and the managed data is stored in a distributed manner; executing ananalyzer configured to perform defect analysis upon receiving a taskrequest, the analyzer comprising a plurality of algorithm serversconfigured to obtain data from the data management platform and performalgorithm analysis on the managed data to derive a result data onunderlying reasons for defects; and executing a data visualization andinteraction interface configured to generate the task requests anddisplay the result data.

In another aspect, the present disclosure provides a method for defectanalysis, comprising obtaining fabrication data information comprisingdefect information; sorting the fabrication data information accordingto fabrication node groups into a plurality of groups of data, arespective one of the plurality of groups of data associated with arespective fabrication node group of the fabrication node groups;calculating weights of evidence for the fabrication node groups toobtain a plurality of weights of evidence, wherein weights of evidencerepresent variability between a percentage of defects in the respectivefabrication node group with respect to a percentage of defects in anentirety of the fabrication node groups; ranking the plurality of groupsof data based on the plurality of weights of evidence; and obtaining alist of plurality of groups of data ranked based on the plurality ofweights of evidence.

Optionally, the respective fabrication node group comprises one or moreselected from a group consisting of a fabrication procedure, a device, asite, and a process section.

Optionally, a respective weight of evidence for the respectivefabrication node group is calculated according to Equation (1):

$\begin{matrix}{{{{woe}_{i} = {{{In}\frac{P\left( y_{i} \right)}{p\left( n_{i} \right)}} = {{In}\frac{\#\mspace{14mu} y_{i}\text{/}\#\mspace{14mu} y_{r}}{\#\mspace{14mu} n_{i}\text{/}\#\mspace{14mu} n_{r}}}}};};} & (1)\end{matrix}$

wherein woe, stands for the respective weight of evidence for therespective fabrication node group; P(yi) stands for a ratio of a numberof positive samples in the respective fabrication node group to a numberof positive samples in all fabrication node groups; P(ni) stands for aratio of a number of negative samples in the respective fabrication nodegroup to a number of negative samples in all fabrication node groups;the positive samples means data comprising defect information associatedwith the respective fabrication node group; the negative samples meansdata in which defect information associated with the respectivefabrication node group is absent; #yi stands for the number of positivesamples in the respective fabrication node group; #yr stands for thenumber of positive samples in all fabrication node groups; #ni standsfor the number of negative samples in the respective fabrication nodegroup; #yr stands for the number of negative samples in all fabricationnode groups.

Optionally, the method further comprises performing feature extractionon parameters of various types to generate parameter featureinformation, wherein one or more of a maximum value, a minimum value, anaverage value, and a median value are extracted for at least one type(e.g., each type) of parameters; and performing data fusion on theparameter feature information and defect information associatedtherewith to obtain a fused data information.

Optionally, the method further comprises extracting the parameterfeature information and the defect information from the fused datainformation; performing a correlation analysis on the parameter featureinformation and the defect information with respect to at least one type(e.g., each type) of parameters from the list of plurality of groups ofdata; generating a plurality of correlation coefficients respectivelyfor a plurality of types of parameters; and ranking absolute values ofthe plurality of correlation coefficients.

Optionally, the plurality of correlation coefficients are a plurality ofpearson correlation coefficients; and a respective pearson correlationcoefficients is calculated according to Equation (2):

$\rho_{x,y} = {\frac{{cov}\mspace{14mu}\left( {x,y} \right)}{\sigma_{x}\sigma_{y}} = {\frac{E\left( {\left( {x - \mu_{x}} \right)\left( {y - \mu_{y}} \right)} \right)}{\sigma_{x}\sigma_{y}} = \frac{{E({xy})} - {{E(x)}{E(y)}}}{\sqrt{{E\left( x^{2} \right)} - {E^{2}(x)}}\sqrt{{E\left( y^{2} \right)} - {E^{2}(y)}}}}}$

wherein x stands for a value for a parameter feature; y stands for avalue for presence or absence of a defect, y is given a value of 1 whenthe defect is present, and y is given a value of 0 when the defect isabsent; μ_(x) stands for a mean value of x; μ_(y) stands for a meanvalue of y; σ_(x)σ_(y) stands for a product of respective standarddeviations of x and y; cov(x,y) stands for a covariance of x,y; andρ(x,y) stands for a respective pearson correlation coefficient.

Optionally, the method further comprises processing the fabrication datainformation to obtain a processed data; wherein processing thefabrication data information comprises performing data fusion onbiographical data information and defect information to obtain a fuseddata information.

In another aspect, the present disclosure provides an intelligent defectanalysis system, comprising a distributed computing system comprisingone or more networked computers configured to execute in parallel toperform at least one common task; one or more computer readable storagemediums storing instructions that, when executed by the distributedcomputing system, cause the distributed computing system to executesoftware modules; wherein the software modules comprise a datamanagement platform configured to intelligently extract, transform, orload raw data from a plurality of data sources into a managed data,wherein the raw data and the managed data comprise defect information,and the managed data is stored in a distributed manner; an analyzerconfigured to perform defect analysis upon receiving a task request, theanalyzer comprising and a plurality of algorithm servers configured toobtain the managed data from the data management platform and performalgorithm analysis on the managed data to derive a result data onunderlying reasons for defects; and a data visualization and interactioninterface configured to generate the task requests and display theresult data; wherein one or more of the plurality of algorithm serversis configured to perform the computer-implemented method describedherein.

BRIEF DESCRIPTION OF THE FIGURES

The following drawings are merely examples for illustrative purposesaccording to various disclosed embodiments and are not intended to limitthe scope of the present invention.

FIG. 1 illustrates a distributed computing environment in someembodiments according to the present disclosure.

FIG. 2 illustrates software modules in an intelligent defect analysissystem in some embodiments according to the present disclosure.

FIG. 3 illustrates software modules in an intelligent defect analysissystem in some embodiments according to the present disclosure.

FIG. 4 illustrates an intelligent defect analysis method using anintelligent defect analysis system in some embodiments according to thepresent disclosure.

FIG. 5 illustrates an intelligent defect analysis method using anintelligent defect analysis system in some embodiments according to thepresent disclosure.

FIG. 6 illustrates an intelligent defect analysis method using anintelligent defect analysis system in some embodiments according to thepresent disclosure.

FIG. 7 illustrates an intelligent defect analysis method using anintelligent defect analysis system in some embodiments according to thepresent disclosure.

FIG. 8 illustrates a data management platform in some embodimentsaccording to the present disclosure.

FIG. 9 depicts a plurality of sub-tables split from a data table storedin a general data layer in some embodiments according to the presentdisclosure.

FIG. 10 illustrates a method of defect analysis in some embodimentsaccording to the present disclosure.

FIG. 11 illustrates a method of defect analysis in some embodimentsaccording to the present disclosure.

DETAILED DESCRIPTION

The disclosure will now be described more specifically with reference tothe following embodiments. It is to be noted that the followingdescriptions of some embodiments are presented herein for purpose ofillustration and description only. It is not intended to be exhaustiveor to be limited to the precise form disclosed.

Various defects may occur in manufacturing of semiconductor electronics.Examples of defects include particle, remain, line defect, hole, splash,wrinkle, discoloration, and bubble. Defects occurring in themanufacturing of semiconductor electronics are difficult to trace. Forexample, engineers may have to rely on manual data sorting to analyzethe root cause of defects based on experience.

In manufacturing a liquid crystal display panel, the fabrication of adisplay panel include at least an array stage, a color filter (CF)stage, a cell stage, and a module stage. In the array stage, a thin filmtransistor array substrate is fabricated. In one example, in the arraystage, a material layer is deposited, the material layer is subject tolithography for example a photoresist is deposited on the materiallayer, the photoresist is subject to exposure and subsequentlydeveloped. Subsequently, the material layer is etched and the remainingphotoresist is removed (“strip”). In the CF stage, a color filtersubstrate is fabricated, involving several steps including coating,exposure, and development. In the cell stage, the array substrate andthe color filter substrate are assembled to form a cell. The cell stageincludes several steps including coating and rubbing an alignment layer,injection of liquid crystal materials, cell sealant coating, cellassembly under vacuum, cutting, grinding, and cell inspection. In themodule stage, peripheral components and circuits are assembled onto thepanel. In one example, the module stage includes several steps includingassembling of a back light, and assembling of a printed circuit board,polarizer attachment, assembling of chip-on-film, assembling ofintegrated circuits, aging, and final inspection.

In manufacturing an organic light emitting diode (OLED) display panel,the fabrication of a display panel include at least four deviceprocesses, include an array stage, an OLED stage, an EAC2 stage, and aModule stage. In the array stage, a back panel of the display panel isfabricated, e.g., including fabrication of a plurality of thin filmtransistors. In the OLED stage, a plurality of light emitting elements(e.g., organic light emitting diodes) are fabricated, an encapsulatinglayer is formed to encapsulate the plurality of light emitting elements,and optionally a protective film is formed on the encapsulating layer.In the EAC2 stage, large glasses are first cut into half glasses, andthen further cut into panels. Moreover, in the EAC2 stage, inspectionequipment is used to inspect the panels to detect defects therein, forexample, dark spots and bright lines. In the Module stage, flexibleprinted circuits are bonded to the panels, e.g., using chip-on-filmtechnology. Cover glass are formed on the surface of the panels.Optionally, further inspections are performed to detect defects in thepanels. The data from the fabrication of display panels includebiographical information, parameter information, and defect information,which are stored in a plurality of data sources. The biographicalinformation is the record information uploaded to the database by eachprocessing equipment from the array stage to the Module stage, includingglass ID, equipment model, site information and so on. The parameterinformation includes data generated by the equipment when processing theglass. Defects may occur in each of the stages. The inspectioninformation may be generated in each of the stages discussed above. Onlyafter the inspection is completed, the inspection information can beuploaded to the database in real time. The inspection information mayinclude defect type and defect position.

In summary, biographical information, parameter information, and defectinformation are obtained using various sensors and inspection equipment.The biographical information, parameter information, and defectinformation are analyzed using the intelligent defect analysis method orsystem, which can quickly determine device, site, and/or stage thatgenerates a defect, providing critical information for subsequentprocess improvement and equipment repair or maintenance, greatlyimproving yield.

Accordingly, the present disclosure provides, inter alia, a datamanagement platform, an intelligent defect analysis system, anintelligent defect analysis method, a computer-program product, and amethod for defect analysis thereof that substantially obviate one ormore of the problems due to limitations and disadvantages of the relatedart. The present disclosure provides an improved data managementplatform having superior functionality. Based on the present datamanagement platform (or other suitable database or data managementplatform), the inventors of the present disclosure further develop anovel and unique intelligent defect analysis system, an intelligentdefect analysis method, a computer-program product, and a method fordefect analysis.

In one aspect, the present disclosure provides an intelligent defectanalysis system. In some embodiments, the intelligent defect analysissystem includes a distributed computing system including one or morenetworked computers configured to execute in parallel to perform atleast one common task; one or more computer readable storage mediumsstoring instructions that, when executed by the distributed computingsystem, cause the distributed computing system to execute softwaremodules. In some embodiments, the software modules include a datamanagement platform configured to store data, and intelligently extract,transform, or load the data, wherein the data comprises at least one ofbiographical data information, parameter information, or defectinformation; an analyzer configured to perform defect analysis uponreceiving a task request, the analyzer including a plurality of businessservers and a plurality of algorithm servers, the plurality of algorithmservers configured to obtain the data directly from the data managementplatform and perform algorithm analysis on the data to derive a resultdata on underlying reasons for defects; and a data visualization andinteraction interface configured to generate the task requests.Optionally, the intelligent defect analysis system is used for defectanalysis in fabrication of display panels. As used herein, the term“distributed computing system” generally refers to an interconnectedcomputer network having a plurality of network nodes that connect aplurality of servers or hosts to one another or to external networks(e.g., the Internet). The term “network node” generally refers to aphysical network device. Example network nodes include routers,switches, hubs, bridges, load balancers, security gateways, orfirewalls. A “host” generally refers to a physical computing deviceconfigured to implement, for instance, one or more virtual machines orother suitable virtualized components. For example, a host can include aserver having a hypervisor configured to support one or more virtualmachines or other suitable types of virtual components.

FIG. 1 illustrates a distributed computing environment in someembodiments according to the present disclosure. Referring to FIG. 1, ina distributed computing environment, a number of autonomouscomputers/workstations, called nodes, are in communication with oneanother in a network, for example, a LAN (Local Area Network), to solvea task, such as execute an application. Each of the computer nodestypically includes its own processor(s), memory and a communication linkto other nodes. The computers can be located within a particularlocation (e.g. cluster network) or can be connected over a large areanetwork (LAN) such as the Internet. In such a distributed computingenvironment, different applications may share information and resources.

The network in the distributed computing environment may include localarea networks (LAN) and wide area networks (WAN). The network mayinclude wired technologies (e.g., Ethernet®) and wireless technologies(e.g., WiFi®, code division multiple access (CDMA), global system formobile (GSM), universal mobile telephone service (UMTS), Bluetooth®,ZigBee®, etc.).

Multiple computing nodes are configured to join a resource group inorder to provide distributed services. A computing node in thedistributed network may include any computing device such as computingdevice or a user device. A computing node may also include data centers.As used herein, a computing node may refer to any computing device ormultiple computing device (i.e., a data center). Software modules may beexecuted on a single computing node (e.g., a server) or distributedacross multiple nodes in any suitable manner.

The distributed computing environment may also include one or morestorage nodes for storing information related to execution of softwaremodules, and/or output generated by execution of software modules,and/or other functions. The one or more storage nodes are incommunication with one another in a network, and are in communicationwith one or more of the computing nodes in the network.

FIG. 2 illustrates software modules in an intelligent defect analysissystem in some embodiments according to the present disclosure.Referring to FIG. 2, the intelligent defect analysis system includes adistributed computing system including one or more networked computersconfigured to execute in parallel to perform at least one common task;one or more computer readable storage mediums storing instructions that,when executed by the distributed computing system, cause the distributedcomputing system to execute software modules. The software modules insome embodiments includes a data management platform DM configured tostore data, and intelligently extract, transform, or load the data; aquery engine QE connected to the data management platform DM andconfigured to obtain the data directly from the data management platformDM; an analyzer AZ connected to the query engine QE and configured toperform defect analysis upon receiving a task request, the analyzer AZincluding a plurality of business servers BS (similar to backendservers) and a plurality of algorithm servers AS, the plurality ofalgorithm servers AS configured to obtain the data directly from thedata management platform DM; and a data visualization and interactioninterface DI configured to generate the task requests. Optionally, thequery engine QE is a query engine based on Impala™ technology. As usedherein, the term “connected to” in the context of the present disclosurerefers to a relationship of having direct information or data flow froma first component of the system to a second component, and/or from thesecond component of the system to the first component.

FIG. 3 illustrates software modules in an intelligent defect analysissystem in some embodiments according to the present disclosure.Referring to FIG. 3, the data management platform DM in some embodimentsincludes an ETL module ETLP configured to extract, transform, or loaddata from a plurality of data sources DS onto a data mart DMT and ageneral data layer GDL. Upon receiving an assigned task, a respectiveone of the plurality of algorithm servers AS is configured to obtain afirst data directly from the data mart DMT. Upon performing defectanalysis, the respective one of the plurality of algorithm servers AS isconfigured to transmit a second data directly to the general data layerGDL. Optionally, the first data is data input into the plurality ofalgorithm servers AS. Optionally, the second data is data output fromthe plurality of algorithm servers AS as a result of computation. Theplurality of algorithm servers AS deploy various common algorithms fordefect analysis, e.g., algorithms based on big data analysis (forexample, algorithms based on specific machine learning models, such asany one or any combination of decision trees, random forests,gradien-boosted decision trees, XGBoost, plain Bayes, support vectormachines, Adaboost, and neural network models; or algorithms based onother statistical algorithmic models such as weight of evidence andinformation value models, and Apriori). The plurality of algorithmservers AS are configured to analyze the data to identify the causes ofthe defects. As used herein, the term “ETL module” refers to a computerprogram logic configured to provide functionality such as extracting,transforming, or loading data. In some embodiments, the ETL module isstored on a storage node, loaded into a memory, and executed by aprocessor. In some embodiments, the ETL module is stored on one or morestorage nodes in a distributed network, loaded into one or more memoryin the distributed network, and executed by one or more processors inthe distributed network.

The data management platform DM stores data for the intelligent defectanalysis system. For example, the data management platform DM storesdata needed for algorithm analysis by the plurality of algorithm serversAS. In another example, the data management platform DM stores resultsof algorithm analysis. The data management platform DM in someembodiments includes the plurality of data sources DS (e.g., data storedin oracle databases), the ETL module ETLP, a data mart DMT (e.g., a datamart based on Apache Hbase™ technology), and the general data layer GDL(e.g., a data storage based on Apache Hive™ technology). For algorithmanalysis and interactive display to a user, the data from the pluralityof data sources DS are cleansed and consolidated into validated data bythe ETL module ETLP. Examples of useful data for defect analysis includetracking history data, dv parameter data, map defect position data, andso on. The amount of data in a typical manufacturing process (e.g., ofdisplay panels) is huge, for example, there might be over 30 millionitems of dv parameter data each day in a typical manufacturing site. Tomeet the user's demand for defect analysis, it is necessary to increasethe speed of reading production data by the algorithm server. In oneexample, the data required for algorithm analysis is stored in a datamart based on Apache Hbase™ technology to improve efficiency and savestorage space. In another example, results of algorithm analysis andother auxiliary data are stored in a general data layer based on ApacheHive™ technology.

Apache Hive™ is an open source data warehouse system built on top ofHadoop used for querying and analyzing large data in form of structuredand semi-structured stored in Hadoop files. Apache Hive™ is mainly usedfor batch processing and thus is known as OLAP. Apache Hive™ is not adatabase and has schema model.

Apache Hbase™ is a non-relational column-oriented distributed databasewhich runs on the top of Hadoop distributed file system (HDFS). Moreoverit is a NoSQL open source database that stores data in columns. ApacheHbase™ is mainly used for transactional processing and known as OLTP.However Real time processing is possible in case of Apache Hbase™.Apache Hbase™ is a type of NoSQL database and is free from schema model.

In one example, various components of the data management platform(e.g., the general data layer, the data warehouse, the data source) maybe in form of a distributed data storage cluster, e.g., based on ApacheHadoop and/or Apache Hive™.

FIG. 8 illustrates a data management platform in some embodimentsaccording to the present disclosure. Referring to FIG. 8, in someembodiments, the data management platform includes a distributed storagesystem (DFS), such as Hadoop Distributed File System (HDFS). The datamanagement platform is configured to collect data generated in a factoryproduction process from a plurality of data sources DS. The datagenerated in the factory production process is stored in a relationaldatabase (e.g., oracle), e.g., using a RDBMS (Relational DatabaseManagement System) grid computing technique. In the RDBMS gridcomputing, a problem that requires very large amounts of computer poweris divided into many small parts, which are distributed to manycomputers for processing. The results of distributed computing arecombined to obtain the final result. For example, in an Oracle RAC (RealApplication Cluster), all servers have direct access to all the data inthe database. RDBMS grid computing based applications, however, havelimited hardware scalability. When the amount of data reaches a certainorder of magnitude, the input/output bottleneck of the hard disk makesit very inefficient to process large amounts of data. The parallelprocessing of the distributed file system can meet the challengepresented by the demand of increasing data storage and computing. In theprocess of intelligent defect analysis, first extracting the data fromthe plurality of data sources DS into the data management platformgreatly expedite the process.

In some embodiments, the data management platform includes a pluralityof groups of data having different contents and/or storage structure. Insome embodiments, the ETL module ETLP is configured to extract raw datafrom a plurality of data sources DS into the data management platform,forming a first data layer (e.g., a data lake DL). The data lake DL is acentralized HDFS or kudu database that is configured to store anystructure or unstructured data. Optionally, the data lake DL isconfigured to store a first group of data extracted by the ETL moduleETLP from a plurality of data sources DS. Optionally, the first group ofdata and the raw data have a same content. The dimension and attributesof the raw data are preserved in the first group of data. In someembodiments, the first group of data stored in the data lake isdynamically updated. Optionally, the first group of data includes areal-time updated data stored in a Kudu™-based database, or aperiodically updated data stored in a Hadoop distributed file system. Inone example, the periodically updated data stored in the Hadoopdistributed file system is a periodically updated data stored in astorage based on Apache Hive™. Optionally, the real-time updated dataincludes data updated at rates below the minute level (for example,excluding the minute level). Optionally, the periodically updated dataincludes data uploaded above the minute level (for example, includingthe minute level).

In some embodiments, the data management platform includes a second datalayer, e.g., the data warehouse DW. The data warehouse DW includes aninternal storage system configured to provide data in abstracted mannersuch as in a table format or a View format, without exposing the filesystem. The data warehouse DW may be based on Apache Hive™. The ETLmodule ETLP is configured to extract, cleanse, transform, or load thefirst group of data to form a second group of data. Optionally, thesecond group of data is formed by subjecting the first group of data tocleansing and standardization.

In some embodiments, the data management platform includes a third datalayer (e.g., a general data layer GDL). The general data layer GDL maybe based on Apache Hive™. The ETL module ETLP is configured to performdata fusion on the second group of data, thereby forming a third groupof data. In one example, the third group of data is a data resultingfrom subjecting the second group of data to data fusion. Examples ofdata fusion include concatenation based on a same field in multipletables. Examples of data fusion further include generation of statisticsof a same field or record (e.g., summation and percentage calculation).In one example, generation of statistics includes counting a number ofdefective panels in a glass, and a percentage of defective panels amonga plurality of panels in a same glass. Optionally, the general datalayer GDL is based on Apache Hive™. Optionally, the general data layerGDL is used for data query.

In some embodiments, the data management platform includes a fourth datalayer (e.g., at least one data mart). In some embodiments, the at leastone data mart include a data mart DMT. Optionally, the data mart DMT isa database of NoSQL type storing information available for computationalprocessing. Optionally, the data mart DMT is based on Apache Hbase™.Optionally, the data mart DMT is used for computation. The ETL moduleETLP is configured to layerize the third data layer to form a fourthgroup of data having a multi-layer index structure. The fourth group ofdata categorizes data based on different types and/or rules, therebyforming the multi-layer index structure. Optionally, the fourth group ofdata having the multi-layer index structure includes a plurality ofsub-tables have index relationship. In one example, categorizing databased on different types and/or rules includes categorizing data basedon various environmental parameters, keys/values, column families, andso on. The first index in the multi-layer index structure corresponds tofiltering criteria of a front-end interface, e.g., corresponds touser-defined analysis criteria in an interactive task sub-interface incommunication with the data management platform, facilitating a moreexpedited data query and computation process.

In some embodiments, the first group of data, the second group of data,the third group of data, and/or the fourth group of data can beconfigured to perform data storage and data query based on one or moredata tables.

In some embodiments, in the process of converting the third group ofdata into the fourth group of data, the data in the general data layerGDL can be imported into the data mart DMT. In one example, a firsttable is generated in the data mart DMT, and a second table (e.g., anexternal table) is generated in the general data layer GDL. The firsttable and the second table are configured to be synchronized so thatwhen data is written into the second table, the first table will besimultaneously updated to include corresponding data.

In another example, a distributed computing processing module may beused for reading data written onto the general data layer GDL. HadoopMapReduce module may be used as the distributed computing processingmodule for reading data written onto the general data layer GDL. Thedata written onto the general data layer GDL may then be written ontothe data mart DMT. In one example, the data may be written into the datamart DMT using a HBase Api. In another example, the Hadoop MapReducemodule, once read the data written onto the data mart DMT, can generateHFile, which is Bulkloaded onto the data mart DMT.

In some embodiments, data flow, data transformation, and data structureamong various components of the data management platform are describedherein. In some embodiments, raw data collected by the plurality of datasources DS includes at least one of biographical data information,parameter information, or defect information. The raw data optionallymay contain dimension information (time, plant, equipment, operator,Map, chamber, Slot, etc.) and attribute information (plant location,equipment age, number of bad points, exception parameters, energyconsumption parameters, process duration, etc.).

Biographical data information contains information of specific processesa product (such as a panel or a glass) is subject to during themanufacturing. Examples of specific processes a product is subject toduring the manufacturing include factory, process, site, device,chamber, card slot, and operator.

Parameter information contains information of information of specificenvironmental parameters and changes thereof a product (such as a panelor a glass) is subject to during the manufacturing. Examples of specificenvironmental parameters and changes thereof a product is subject toduring the manufacturing include environmental particle condition,device temperature, and device pressure.

Defect information contains information of product quality based uponinspection. Examples product quality information include defect type,defect position, and defect dimension.

In some embodiments, parameter information includes device parameterinformation. Optionally, device parameter information includes at leastthree types of data, which may be exported from a General Model forCommunications and Control of Manufacturing Equipment (GEM) interface. Afirst type of data that can be exported from a GEM interface is datavariable (DV), which can be collected as the event occurs. Thus, thedata variable is only valid in the context of the event. In one example,the GEM interface can provide an event called PPChanged, which istriggered when a recipe is changed; and a data variable named “changedrecipe”, which is only valid in the context of the PPChanged event.Polling this value at other times may have invalid or unexpected data. Asecond type of data that can be exported from a GEM interface is statusvariable (SV), which contains device specific information that is validat any time. In one example, the device may be a temperature sensor, andthe GEM interface provides temperature status variable of one or moremodules. The host can request a value of this status variable at anytime, and can expect that the value to be true. A third type of datathat can be exported from a GEM interface is device constant (EC), whichcontains data items set by the device. The device constant determinesthe behavior of the device. In one example, the GEM interface provides adevice constant name “MaxSimultaneousTraces” that specifies the maximumnumber of traces that can be requested from the host at the same time.The value of the device constant is always guaranteed to be valid andup-to-date.

In some embodiments, the data lake DL is configured to store a firstgroup of data formed by extracting raw data from a plurality of datasources by the ETL module ETLP, the first group of data having samecontents as the raw data. The ETL module ETLP is configured to extractthe raw data from the plurality of data sources DS while maintaining thedimension information (e.g., dimension columns) and the attributeinformation (e.g., attribute columns). The data lake DL is configured tostore the extracted data arranged according to time of extraction. Thedata may be stored in the data lake DL with a new name indicating “datalake” and/or attribute(s) of respective data sources, while maintainingthe dimension and attributes of the raw data. The first group of dataand the raw data are stored in different forms. The first group of datais stored in a distributed file system, while the raw data is stored ina relational database such as an Oracle database. In one example, thebusiness data collected by the plurality of data sources DS includesdata from various business systems, including, for example, yieldmanagement system (YMS), fault detection and classification (FDC)system, and manufacturing execution system (MES). The data in thesebusiness systems have their respective signatures, such as productmodel, production parameters and equipment model data. The ETL moduleETLP, using tools such as sqoop command, number stack tool, pentahotool, extracts raw production data from each of the business systemsinto hadoop in the original data format, thereby achieving convergenceof data from multiple business systems. The extracted data are stored inthe data lake DL. In another example, the data lake DL is based ontechnologies such as Hive™ and Kudu™. The data lake DL containsdimension columns (time, plant, equipment, operator, Map, chamber, Slot,etc.) and attribute columns (plant location, equipment age, number ofbad points, exception parameters, energy consumption parameters, processduration, etc.) involved in the factory automation process.

In one example, the present data management platform integrates variousbusiness data (e.g., data associated with semiconductor electronicsmanufacturing) into the plurality of data sources DS (e.g., Oracledatabases). The ETL module ETLP extracts the data from the plurality ofdata sources DS into the data lake DL, for example, using a number stacktool, a SQOOP tool, a kettle tool, a Pentaho tool, or a DataX tool. Thedata is then cleansed, transformed and loaded into the data warehouse DWand the general data layer GDL. The data warehouse DW, the general datalayer GDL, and the data mart DMT store huge amount of data andanalytical results, utilizing tools such as Kudu™, Hive™, and Hbase™.

Information generated in various stages of the fabrication process isobtained by various sensors and inspection equipment, and subsequentlysaved in the plurality of data sources DS. Computation and analysisresults generated by the present intelligent defect analysis system arealso saved in the plurality of data sources DS. Data synchronization(flow of data) among the various components of the data managementplatform is realized through the ETL module ETLP. For example, the ETLmodule ETLP is configured to obtain parameter configuration templates ofsynchronized processes, including network permissions and database portconfiguration, in-flow data library name and table names, out-flow datalibrary name and table names, field correspondence, task type,scheduling cycle, and so on. The ETL module ETLP configures parametersto the synchronized processes based on the parameter configurationtemplates. The ETL module ETLP synchronizes data and cleansessynchronized data based on process configuration templates. The ETLmodule ETLP cleanses the data through SQL statements to remove null,remove outliers, and establish correlation between related tables. Datasynchronization tasks include data synchronization between the pluralityof data sources DS and the data management platform, and datasynchronization among various layers (e.g., the data lake DL, the datawarehouse DW, the general data layer GDL, or the data mart DMT) of thedata management platform.

In another example, data extraction to the data lake DL may be done inreal time or offline. In the offline mode, the data extraction tasks arescheduled periodically. Optionally, in the offline mode, the extracteddata may be stored in a storage based on Hadoop distributed file system(e.g., a Hive™-based database). In the real-time mode, the dataextraction tasks may be performed by OGG (Oracle GoldenGate) incombination with Apache Kafka. Optionally, in the real time mode, theextracted data may be stored in Kudu™-based database. OGG reads the logfiles in the plurality of data sources (e.g., oracle database) to getthe add/delete data. In another example, the topic information is readby flink, json is selected as the synchronized field type. The data isparsed using jar package, and the parsed information is transmitted tokudu api to realize the add/delete of kudu table data. In one example, afront-end interface may perform displaying, querying, and/or analysisbased on data stored in the Kudu™-based database. In another example,the front-end interface may perform displaying, querying, and/oranalysis based on data stored in any one or any combination of theKudu™-based database, a Hadoop distributed file system (e.g., an ApacheHive™-based database), and/or an Apache Hbase™-based database. Inanother example, short term data (e.g., generated within several months)is stored in the Kudu™-based database, and long term data (e.g., anentirety of data generated in all periods) is stored in a Hadoopdistributed file system (e.g., the Apache Hive™-based database). Inanother example, the ETL module ETLP is configured to extract datastored in the Kudu™-based database into the Hadoop distributed filesystem (e.g., the Apache Hive™-based database).

The data warehouse DW is built based on the data lake DL, by combing thedata from various business systems (MDW, YMS, MES, FDC, etc.). The dataextracted from the data lake DL is partitioned according to taskexecution time, which does not fully match the time stamp in the rawdata. In addition, there is a possibility of data duplication. Thus, itis necessary to build the data warehouse DW based on the data lake DL,by cleaning and standardizing the data in the data lake DL to meet theneeds of upper layer applications for data accuracy and partitioning.The data tables stored in the data warehouse DW is obtained bysubjecting the data in the data lake DL to cleaning and standardization.Based on user requirements, the field format is standardized to ensurethat the data tables in the data warehouse DW are completely consistentwith that in the plurality of data sources DS. At the same time, thedata is partitioned by day or month according to time and other fields,greatly improving query efficiency and reducing running memoryrequirement. The data warehouse DW may be one or any combination of theKudu™-based database and an Apache Hive™-based database.

In some embodiments, the ETL module ETLP is configured to cleanse theextracted data stored in the data lake into the cleansed data, and thedata warehouse is configured to store the cleansed data. Examples ofcleansing performed by the ETL module ETLP include removal of redundantdata, removal of null data, remove of dummy field, and so on.

In some embodiments, the ETL module ETLP is further configured toperformed standardization (e.g., field standardization and formatstandardization) on the extracted data stored in the data lake, and thecleansed data are data subject to the field format standardization(e.g., format standardization of date and time information).

In some embodiments, at least a portion of the business data in theplurality of data sources DS is in a binary large object (blob) format.After the data extraction, at least a portion of the extracted datastored in the data lake DL is in a compressed hexadecimal format.Optionally, at least a portion of the cleansed data stored in the datawarehouse DW is obtained by decompressing and processing the extracteddata. Optionally, the data in the blob format is converted in thehexadecimal format when extracted and stored in the data lake DL; thedata in the hexadecimal format is decompressed when extracted and storedin the data warehouse DW, thereby forming the second group of data. Inone example, the business systems (e.g., FDC system discussed above) areconfigured to store a huge amount of parameter data. Thus, the data hasto be compressed into the blob format in the business systems. Duringdata extraction (e.g., from the oracle database to the hive database),the blob field will be converted into a hexadecimal (HEX) string. Toretrieve the parameter data stored in the file, the HEX file isdecompressed and the contents of the file can be obtained directlythereafter. The required data is coded to form a long string, and thedifferent contents are split by specific symbols, depending on outputrequirements. To obtain data in the required format, the long string issubject to operations such as cutting according to special charactersand row-column conversion. The processed data is written into the targettable (e.g., data in a table format stored in the data warehouse DWdiscussed above) along with the original data.

In one example, the cleansed data stored in the data warehouse DWmaintains the dimension information (e.g., dimension columns) and theattribute information (e.g., attribute columns) of the raw data in theplurality of data sources DS. In another example, the cleansed datastored in the data warehouse DW maintains a same data table name as thatin the plurality of data sources DS.

In some embodiments, the ETL module ETLP is further configured togenerate a dynamically updated table that is automatically updatedperiodically. Optionally, a general data layer GDL is configured tostore the dynamically updated table comprising information on defects ofhigh occurrence, as discussed above. In one example, the information ondefects of high occurrence refers to the information on the top five orthe top ten defects ranked by occurrence. In another example, theinformation on defects of high occurrence refers to defect informationspecified by a user. Optionally, the data mart DMT is configured tostore the dynamically updated table comprising information on defects ofhigh occurrence, as discussed above.

The general data layer GDL is built based on the data warehouse DW. Insome embodiments, the GDL is configured to store a third group of dataformed by subjecting the second group of data to data fusion by the ETLmodule ETLP. Optionally, the data fusion are performed based ondifferent themes. The data in the general data layer GDL are highlythemed and highly aggregated, greatly improving query speed. In oneexample, tables having correlation constructed according to differentuser needs or different themes may be built using the tables in the datawarehouse DW, the tables being assigned names according to theirrespective utilities.

Various themes may correspond to different data analysis needs. Forexample, themes may correspond to different defect analysis needs. Inone example, a theme may correspond to analysis of defects attributed toone or more fabrication node groups (e.g., one or more devices), and thedata fusion based on said theme may include data fusion on biographicalinformation of manufacturing process and defect information associatedtherewith. In another example, a theme may correspond to analysis ofdefects attributed to one or more parameter types, and the data fusionbased on said theme may include data fusion on parameter featureinformation and defect information associated therewith. In anotherexample, a theme may correspond to analysis of defects attributed to oneor more device operations (e.g., device defined by a respectiveoperation site at which the respective device perform a respectiveoperation), and the data fusion based on said theme may include datafusion on parameter feature information, biographical information ofmanufacturing process, and defect information associated therewith. Inanother example, a theme may correspond to feature extraction onparameters of various types to generate parameter feature information,wherein one or more of a maximum value, a minimum value, an averagevalue, and a median value are extracted for at least one type (e.g.,each type) of parameters. In one example, the parameter featureinformation includes at least one of device parameter information suchas temperature, humidity, pressure, and so on, as well as data onenvironmental granularity.

In some embodiments, defect analysis includes performing featureextraction on parameters of various types to generate parameter featureinformation; and performing data fusion on at least two of the parameterfeature information, biographical information of a manufacturingprocess, and defect information associated therewith. Optionally,performing data fusion includes performing data fusion on parameterfeature information and defect information associated therewith.Optionally, performing data fusion includes performing data fusion onparameter feature information, biographical information of themanufacturing process, and defect information associated therewith. Inanother example, performing data fusion includes performing data fusionon the parameter feature information and biographical information of themanufacturing process to obtain first fused data information; andperforming data fusion on the first fused data information and defectinformation associated therewith to obtain second fused datainformation. In one example, the second fused data information includesglass serial number, manufacturing site information, device information,the parameter feature information, and the defect information. The datafusion is performed in the general data layer GDL, e.g., by buildingtables having correlation constructed according to user needs or themes.Optionally, the step of performing data fusion includes performing datafusion on the biographical information and the defect information.Optionally, the step of performing data fusion includes performing datafusion on all three of the parameter feature information, biographicalinformation of a manufacturing process, and defect informationassociated therewith.

In one example, the CELL_PANEL_MAIN table in the data warehouse DWstores the basic biographical data of the panel in the cell factory, andthe CELL_PANEL_CT table stores the details of the CT process in thefactory. The general data layer GDL is configured to perform acorrelation operation based on the CELL_PANEL_MAIN table and theCELL_PANEL_CT table, to create a wide table YMS_PANEL. The basicbiographical data of the panel and the details of the CT process can bequeried in the YMS_PANEL table. The YMS prefix in the table name“YMS_PANEL” stands for the themes for defect analysis, and the PANELprefix stands for specific PANEL information stored in the table. Bysubjecting the tables in the data warehouse DW to the correlationoperation by the general data layer GDL, data in different tables can befused and correlated.

According to different business analysis requirements, and based onglass, hglass, and panel, the tables in the general data layer GDL canbe classified into the following datatags: production biographic, defectrate, defect MAP, DV, SV, inspection data, and test data.

The data mart DMT is build based on the data warehouse DW and/or thegeneral data layer GDL. The data mart DMT may be used for providingvarious reporting data and data needed for analysis, particularly highlycustomized data. In one example, customized data provided by the datamart DMT include consolidated data on defect rates, frequency ofspecific defects, and so on. In another example, data in the data lakeDL and the general data layer GDL are stored in Hive-based databases,data in the data mart DMT are stored in Hbase-based databases.Optionally, table names in the data mart DMT can be kept consistent withthose in the general data layer GDL. Optionally, the general data layerGDL is based on Apache Hive™ technology, and the data mart DMT is basedon Apache Hbase™ technology. The general data layer GDL is used for dataquery through a user interface. Data in Hive can be quickly queried inHive through Impala. The data mart DMT is used for computation. Based onthe advantage of columnar data storage in Hbase, the plurality ofalgorithm servers AS can quickly access the data in the Hbase.

In some embodiments, the data mart DMT is configured to store aplurality of sub-tables split from a respective one of the data tablesstored in the general data layer GDL. In some embodiments, the datastored in the data mart DMT and the data stored in the general datalayer GDL have the same contents. The data stored in the data mart DMTand the data stored in the general data layer GDL differ from each otherin that they are stored in different data models. Depending on differenttypes of NoSQL databases used for the data mart DMT, the data in thedata mart DMT may be stored in different data models. Examples of datamodels corresponding to different NoSQL databases include a key-valuedata model, a column family data model, a versioned document data model,and a graph structure data model. In some embodiments, a query to thedata mart DMT may be performed based on specified keys, to quicklylocate the data (e.g., values) to be queried. Accordingly, and as morespecifically discussed below, the table stored in the general data layerGDL may be split into at least three sub-tables in the data mart DMT.The first sub-table corresponds to user-defined analysis criteria in aninteractive task sub-interface. The second sub-table corresponds tospecified keys (e.g., product serial numbers). The third sub-tablecorresponds to values (e.g., the values stored in the table in thegeneral data layer GDL, comprising fused data). In one example, therange of products that need to be analyzed by a user may be determinedbased on the first sub-table; and the serial number (key) in the secondsub-table corresponding to a respective product may be used for queryingthe corresponding data (value) in the third sub-table. In one example,the data mart DMT utilizes a NoSQL database based on the Apache Hbase™technology; the specified keys in the second sub-table may be row keys;and the fused data in the third sub-table may be stored in a columnfamily data model. Optionally, the fused data in the third sub-table maybe fused data from at least two of the parameter feature information,biographical information of a manufacturing process, and defectinformation associated therewith. Moreover, the data mart DMT mayinclude a fourth sub-table. Certain characters in the third sub-tablemay be stored in codes, for example, due to their lengths or otherreasons. The fourth sub-table includes the characters (e.g., devicenames, fabrication sites) corresponding to these codes stored in thethird sub-table. The indexes or queries among the first sub-table, thesecond sub-table, and the third sub-table may be based on the codes. Thefourth sub-table may be utilized to replace the codes with thecharacters before the results are presented to the user interface.

In some embodiments, the plurality of sub-tables have index relationshipbetween at least two sub-tables of the plurality of sub-tables.Optionally, data in the plurality of sub-tables are categorized based ontypes and/or rules. In some embodiments, the plurality of sub-tablesincludes a first sub-table (e.g., an attribute sub-table) comprising aplurality of environmental factors corresponding to user-definedanalysis criteria in an interactive task sub-interface in communicationwith the data management platform; a second sub-table comprising productserial numbers (e.g., glass identification numbers or lot identificationnumbers); and a third sub-table (e.g., a main sub-table) comprisingvalues in the third group of data that correspond to the product serialnumbers. Optionally, based on different themes, the second sub-table mayinclude different specified keys such as the glass identificationnumbers or the lot identification numbers (e.g., multiple secondsub-tables). Optionally, values in the third group of data thatcorrespond to the glass identification numbers through an indexrelationship between the third sub-table and the second sub-table.Optionally, the plurality of sub-tables further includes a fourthsub-table (e.g., a metadata sub-table) comprising values in the thirdgroup of data that correspond to the lot identification numbers.Optionally, the second sub-table further includes lot identificationnumbers; values in the third group of data that correspond to the lotidentification numbers may be obtained through an index relationshipbetween the second sub-table and the fourth sub-table. Optionally, theplurality of sub-tables further includes a fifth sub-table (e.g., a codegenerator sub-table) comprising manufacturing site information anddevice information. Optionally, the third sub-table includes codes orabbreviations for manufacturing site and device, through an indexrelationship between the third sub-table and the fifth sub-table, themanufacturing site information and device information may be obtainedfrom the fifth sub-table.

FIG. 9 depicts a plurality of sub-tables split from a data table storedin a general data layer in some embodiments according to the presentdisclosure. Referring to FIG. 9, in some embodiments, the plurality ofsub-tables include one or more of: an attribute sub-table comprising aplurality of environmental factors corresponding to user-definedanalysis criteria in an interactive task sub-interface in communicationwith the data management platform; a context sub-table comprising atleast first multiple environmental factors of the plurality ofenvironmental factors and multiple manufacture stage factors, andmultiple columns corresponding to second multiple environmental factorsof the plurality of environmental factors; a metadata sub-tablecomprising at least a first manufacture stage factor of the multiplemanufacture stage factors and a device factor associated with a firstmanufacture stage, and multiple columns corresponding to parametersgenerated in the first manufacture stage; a main sub-table comprising atleast a second manufacture stage factor of the multiple manufacturestage factors, and multiple columns corresponding to parametersgenerated in a second manufacture stage; and a code generator sub-tablecomprising at least third multiple environmental factors of theplurality of environmental factors and the device factor.

In one example, the plurality of sub-tables include sub-tables stored inthe column family database, including one or more of the following: anattribute sub-table including a key made up of datatag, factoryinformation, manufacturing site information, product model information,product type information, and product serial number, filtering criteriafor which may be determined based on a selection on an interactive tasksub-interface; a context sub-table including a key made up of firstthree numbers of MD5 encryption site, the factory information, themanufacturing site information, the datatag, manufacture end time, lotserial number, and glass serial number, a first column for the productmodel information, a second column for the product serial number, and athird column for the product type information, the filtering criteriadetermined for the attribute sub-table may be mapped to correspondinglot serial numbers and glass serial numbers through the contextsub-table; a metadata sub-table including a key made up of the firstthree numbers of MD5 encryption site, the lot serial number, thedatatag, the manufacturing site information, and device information, afirst column for manufacturing time, a second column for manufacturingparameter, and corresponding values including specific values, valuescorresponding to the lot serial numbers determined using the contextsub-table may be obtained through the metadata sub-table; a mainsub-table including a key made up of the first three numbers of MD5encryption site, serial number, and the glass serial number, a firstcolumn for the manufacturing time, and a second column for manufacturingparameter, a first column for manufacturing time, a second column formanufacturing parameter, and corresponding values including specificvalues, values corresponding to the glass serial numbers determinedusing the context sub-table may be obtained through the main sub-table;and a code generator sub-table including a key made up of the datatag,the manufacturing site information, and the device information, a columnfor serial number, the corresponding manufacturing site information andthe device information in the code generator sub-table may be obtainedusing the serial number in the main sub-table. Optionally, the pluralityof environmental factors in the attribute sub-table include datatag,factory information, manufacturing site information, product modelinformation, product type information, and product serial number.Optionally, the multiple manufacture stage factors include the lotserial number and the glass serial number. Optionally, the device factorcomprises the device information.

Referring to FIG. 2 and FIG. 3, the software modules in some embodimentsfurther include a load balancer LB connected to the analyzer AZ.Optionally, the load balancer LB (e.g., a first load balancer LB1) isconfigured to receive task requests and configured to assign the taskrequests to one or more of the plurality of business servers BS toachieve load balance among the plurality of business servers BS.Optionally, the load balancer LB (e.g., a second load balancer LB2) isconfigured to assign tasks from the plurality of business servers BS toone or more of the plurality of algorithm servers AS to achieve loadbalance among the plurality of algorithm servers AS. Optionally, theload balancer LB is a load balancer based on Nginx™ technology.

In some embodiments, the intelligent defect analysis system isconfigured to meet demands of many users simultaneously. By having theload balancer LB (e.g., the first load balancer LB1), the system sendsuser requests to the plurality of business servers AS in a balancedmanner, keeping the overall performance of the plurality of businessservers AS optimal and preventing the slow response of services due toexcessive pressure on a single server.

Similarly, by having the load balancer LB (e.g., the second loadbalancer LB2), the system sends tasks to the plurality of algorithmservers AS in a balanced manner, keeping the overall performance of theplurality of algorithm servers AS optimal. In some embodiments, whendesigning the load balancing strategy, not only the number of tasks sentto each of the plurality of algorithm servers AS should be considered,but also the amount of computational burden required by each task. Inone example, three types of tasks are involved, including defectanalysis of a type “glass”, defect analysis of a type “hglass”, anddefect analysis of a type “panel”. In another example, a number ofdefect data items associated with the type “glass” is 1 million per weekon average, and a number of defect data items associated with the type“panel” is 30 million per week on average. Thus, the amount ofcomputational burden required defect analysis of the type “panel” is fargreater than the amount of computational burden required defect analysisof the type “glass”. In another example, the load balancing is performedusing a formula f (x, y, z)=mx+ny+oz, wherein x stands for the number oftasks for defect analysis of the type “glass”; y stands for the numberof tasks for defect analysis of the type “hglass”; z stands for thenumber of tasks for defect analysis of the type “panel”; m stands for aweight assigned for defect analysis of the type “glass”; n stands for aweight assigned for defect analysis of the type “hglass”; and o standsfor a weight assigned for defect analysis of the type “panel”. Theweights are assigned based on the amount of computational burdenrequired defect analysis of at least one type (e.g., each type).Optionally, m+n+o=1.

In some embodiments, the ETL module ETLP is configured to generate adynamically updated table that is automatically updated periodically(e.g., every day, every hour, etc.). Optionally, the general data layerGDL is configured to store the dynamically updated table. In oneexample, the dynamically updated table is generated based on the logicof calculating the incidence of defects in a factory. In anotherexample, data from multiple tables in the data management platform DMare consolidated and subject to various calculation to generate thedynamically updated table. In another example, the dynamically updatedtable includes information such as job name, defect code, occurrencefrequency of defect code, the level of the defect code(glass/hglass/panel), factory, product model, date and otherinformation. The dynamically updated table is updated regularly, whenthe production data in the data management platform DM changes, theinformation in the dynamically updated table will be updatedaccordingly, so as to ensure that the dynamically updated table can haveall the factory's defect code information.

FIG. 4 illustrates an intelligent defect analysis method using anintelligent defect analysis system in some embodiments according to thepresent disclosure. Referring to FIG. 4, in some embodiments, the datavisualization and interaction interface DI is configured to generate atask request; the load balancer LB is configured to receive the taskrequest and configured to assign the task request to one or more of theplurality of business servers to achieve load balance among theplurality of business servers; the one or more of the plurality ofbusiness servers are configured to transmit a query task request to thequery engine QE; the query engine QE, upon receiving the query taskrequest from the one or more of the plurality of business servers, isconfigured to query the dynamically updated table to obtain informationon defects of high occurrence, and transmit the information on defectsof high occurrence to one or more of the plurality of business servers;the one or more of the plurality of business servers are configured totransmit defect analysis tasks to the load balancer LB for assigning thedefect analysis tasks to the one or more of the plurality of algorithmservers to achieve load balance among the plurality of algorithmservers; upon receiving the defect analysis tasks, the one or more ofthe plurality of algorithm servers are configured to obtain the datadirectly from the data mart DMT to perform defect analysis; and uponcompletion of the defect analysis, the one or more of the plurality ofalgorithm servers are configured to transmit results of the defectanalysis to the general data layer GDL.

The query engine QE enables fast access to the data management platformDM, e.g., reading and writing data quickly to or from the datamanagement platform DM. As compared to direct query through a generaldata layer GDL, having the query engine QE is advantageous as itobviates the need of executing a map reduce (MR) program in order toquery the general data layer GDL (e.g., Hive data storage). Optionally,the query engine QE may be a distributed query engine that can query thegeneral data layer GDL (HDFS or Hive) in real time, greatly reducinglatency and improving the responsiveness of the entire system. The queryengine QE may be implemented using various appropriate technologies.Examples of technologies for implementing the query engine QE includeImpala™ technology, Kylin™ technology, Presto™ technology, andGreenplum™ technology.

In some embodiments, the task request is an automatically recurring taskrequest, the automatically recurring task request defining a recurringperiod for which the defect analysis is to be performed. FIG. 5illustrates an intelligent defect analysis method using an intelligentdefect analysis system in some embodiments according to the presentdisclosure. Referring to FIG. 5, in some embodiments, the datavisualization and interaction interface DI is configured to generate anautomatically recurring task request; the load balancer LB is configuredto receive the automatically recurring task request and configured toassign the automatically recurring task request to one or more of theplurality of business servers to achieve load balance among theplurality of business servers; the one or more of the plurality ofbusiness servers are configured to transmit a query task request to thequery engine QE; the query engine QE, upon receiving the query taskrequest from the one or more of the plurality of business servers, isconfigured to query the dynamically updated table to obtain informationon defects of high occurrence limited to the recurring period, andtransmit the information on defects of high occurrence to one or more ofthe plurality of business servers; upon receiving the information ondefects of high occurrence during the recurring period, the one or moreof the plurality of business servers are configured to generate thedefect analysis tasks based on the information on defects of highoccurrence during the recurring period; the one or more of the pluralityof business servers are configured to transmit defect analysis tasks tothe load balancer LB for assigning the defect analysis tasks to the oneor more of the plurality of algorithm servers to achieve load balanceamong the plurality of algorithm servers; upon receiving the defectanalysis tasks, the one or more of the plurality of algorithm serversare configured to obtain the data directly from the data mart DMT toperform defect analysis; and upon completion of the defect analysis, theone or more of the plurality of algorithm servers are configured totransmit results of the defect analysis to the general data layer GDL.

Referring to FIG. 3, the data visualization and interaction interface DIin some embodiments includes an automatic task sub-interface SUB1allowing input of the recurring period for which the defect analysis isto be performed. The automatic task sub-interface SUB1 enables automaticdefect analysis of defects of high occurrence periodically. In theautomatic task mode, the information on defects of high occurrence istransmitted to the plurality of algorithm servers AS for analyzing theunderlying reasons for causing the defects. In on example, a user setsup the recurring period for which the defect analysis is to be performedin the automatic task sub-interface SUB1. The query engine QE regularlycaptures the defect information from the dynamically updated table basedon the system settings, and sends the information to the plurality ofalgorithm servers AS for analysis. In this way, the system canautomatically monitor the defects of high occurrence, and thecorresponding analysis results can be stored in a cache ready to beaccessed for display in the data visualization and interaction interfaceDI.

In some embodiments, the task request is an interactive task request.FIG. 6 illustrates an intelligent defect analysis method using anintelligent defect analysis system in some embodiments according to thepresent disclosure. Referring to FIG. 6, in some embodiments, the datavisualization and interaction interface DI is configured to receive auser-defined analysis criteria, and configured to generate theinteractive task request based on the user-defined analysis criteria;the data visualization and interaction interface DI is configured togenerate an interactive task request; the load balancer LB is configuredto receive the interactive task request and configured to assign theinteractive task request to one or more of the plurality of businessservers to achieve load balance among the plurality of business servers;the one or more of the plurality of business servers are configured totransmit a query task request to the query engine; the query engine QE,upon receiving the query task request from the one or more of theplurality of business servers, is configured to query the dynamicallyupdated table to obtain information on defects of high occurrence, andtransmit the information on defects of high occurrence to one or more ofthe plurality of business servers; upon receiving the information ondefects of high occurrence, the one or more of the plurality of businessservers are configured to transmit the information to the datavisualization and interaction interface; the data visualization andinteraction interface DI is configured to display the information ondefects of high occurrence and a plurality of environmental factorsassociated with the defects of high occurrence, and configured toreceive a user-defined selection of one or more environmental factorsfrom a plurality of environmental factors, and transmit the user-definedselection to the one or more of the plurality of business servers; theone or more of the plurality of business servers are configured togenerate the defect analysis tasks based on the information and theuser-defined selection; the one or more of the plurality of businessservers are configured to transmit defect analysis tasks to the loadbalancer LB for assigning the defect analysis tasks to the one or moreof the plurality of algorithm servers to achieve load balance among theplurality of algorithm servers; upon receiving the defect analysistasks, the one or more of the plurality of algorithm servers areconfigured to obtain the data directly from the data mart DMT to performdefect analysis; and upon completion of the defect analysis, the one ormore of the plurality of algorithm servers are configured to transmitresults of the defect analysis to the general data layer GDL.

Referring to FIG. 3, the data visualization and interaction interface DIin some embodiments includes an interactive task sub-interface SUB2allowing input of the user-defined analysis criteria including theuser-defined selection of one or more environmental factors. In oneexample, the user may filter various environmental factors,level-by-level, including data source, factory, manufacturing site,model, product model, lot, etc. in the interactive task sub-interfaceSUB2. The one or more of the plurality of business servers BS areconfigured to generate the defect analysis tasks based on theinformation on defects of high occurrence and the user-defined selectionof one or more environmental factors. The analyzer AZ interacts with thegeneral data layer GDL continuously, and causes the selected one or moreenvironmental factors to be displayed on the interactive tasksub-interface SUB2. The interactive task sub-interface SUB2 allows auser, based on the user's experience, limit the environmental factors toa few, for example, certain selected equipment or certain selectedparameters.

In some embodiments, the general data layer GDL is configured togenerate tables based on different themes. In one example, the tablesinclude a tracking table containing biographical information, whichcontains information of sites and devices that glass or panel has passedthrough during the entire fabrication process. In another example, thetables include a dv table containing parameter information uploaded bythe devices. In another example, if the user only wants to analyzeequipment correlation, the user can select the tracking table foranalysis. In another example, if the user only wants to analyze theequipment parameters, the user can select the dv table for analysis.

Referring to FIG. 3, the analyzer AZ in some embodiments furtherincludes a cache server CS. Optionally, the cache server CS isconfigured to store a portion of results of previously performed defectanalysis tasks in a cache C. In some embodiments, the data visualizationand interaction interface DI further includes a defect visualizationsub-interface SUB-3. In one example, a main function of the defectvisualization sub-interface SUB-3 is to allow a user to customize thequery and display corresponding results of previously performed defectanalysis tasks upon a user clicking on a defect code. In one example,the user clicks on the defect code and the system sends the request toone or more of the plurality of business servers BS via the loadbalancer LB. The one or more of the plurality of business servers BSfirst queries result data cached in the cache C, and the system displaysthe cached result data directly if it exists. If the result datacorresponding to the selected defect code is not presently cached in thecache C, the query engine QE is configured to query the general datalayer GDL for the result data corresponding to the selected defect code.Once queried, the system caches the result data corresponding to theselected defect code in the cache C, which may be available for a nextquery on the same defect code.

FIG. 7 illustrates an intelligent defect analysis method using anintelligent defect analysis system in some embodiments according to thepresent disclosure. Referring to FIG. 7, in some embodiments, the defectvisualization sub-interface DI is configured to receive a user-definedselection of a defect to be analyzed and generate a call request; theload balancer LB is configured to receive the call request andconfigured to assign the call request to one or more of the plurality ofbusiness servers to achieve load balance among the plurality of businessservers; the one or more of the plurality of business servers isconfigured to transmit the call request to the cache server; and thecache server is configured to determine whether information on thedefect to be analyzed is stored in the cache. Optionally, upon adetermination that the information on the defect to be analyzed isstored in the cache, the one or more of the plurality of businessservers is configured to transmit the information on the defect to beanalyzed to the defect visualization sub-interface for displaying.Optionally, upon a determination that the information on the defect tobe analyzed is not stored in the cache, the one or more of the pluralityof business servers is configured to transmit a query task request tothe query engine; the query engine, upon receiving the query taskrequest from the one or more of the plurality of business servers, isconfigured to query the dynamically updated table to obtain informationon the defect to be analyzed, and transmit the information on the defectto be analyzed to the cache; the cache is configured to store theinformation on the defect to be analyzed; and the one or more of theplurality of business servers is configured to transmit the informationon the defect to be analyzed to the defect visualization sub-interfacefor displaying.

Optionally, the portion of results of previously performed defectanalysis tasks includes results of previously performed defect analysistasks based on automatically recurring task requests. Optionally, theportion of results of previously performed defect analysis tasksincludes results of previously performed defect analysis tasks based onautomatically recurring task requests; and results of previouslyperformed defect analysis tasks obtained based on the query taskrequest.

By having the cache server CS, high demand on the response speed of thesystem (e.g., displaying results associated with a defect code) can bemet. In one example, up to as much as 40 tasks may be generated everyhalf an hour by the automatic recurring task requests, with each taskassociated with up to five different defect codes, and each defect codeassociated with up to 100 environmental factors. If all the analysisresults are cached, a total number of 40*5*100=20,000 queries will haveto be stored in the cache C, which will be puts a lot of pressure oncluster memory. In one example, the portion of results of previouslyperformed defect analysis tasks are limited to results associated withthe top three highest ranked defect codes, and only this portion iscached.

Various appropriate methods for defect analysis may be implemented byone or more of the plurality of algorithm servers of the intelligentdefect analysis system described herein. FIG. 10 illustrates a method ofdefect analysis in some embodiments according to the present disclosure.Referring to FIG. 10, in some embodiments, the method includes obtainingfabrication data information comprising defect information; sorting thefabrication data information according to fabrication node groups into aplurality of groups of data, a respective one of the plurality of groupsof data associated with a respective fabrication node group of thefabrication node groups; calculating weights of evidence for thefabrication node groups to obtain a plurality of weights of evidence,wherein weights of evidence represent variability between a percentageof defects in the respective fabrication node group with respect to apercentage of defects in an entirety of the fabrication node groups;ranking the plurality of groups of data based on the plurality ofweights of evidence; obtaining a list of plurality of groups of dataranked based on the plurality of weights of evidence; and performingdefect analysis on one or more selected groups of the plurality ofgroups of data. Optionally, the respective fabrication node groupcomprises one or more selected from a group consisting of a fabricationprocedure, a device, a site, and a process section. Optionally, thefabrication data information may be obtained from the data mart DMT.Optionally, the fabrication data information may be obtained from thegeneral data layer GDL.

Optionally, the method includes processing fabrication data informationcomprising biographical data information, defect information, to obtaina processed data; sorting the processed data according to equipmentgroups into a plurality of groups of data, a respective one of theplurality of groups of data associated with a respective equipment groupof the equipment groups; calculating weights of evidence for theequipment groups to obtain a plurality of weights of evidence; rankingthe plurality of groups of data based on the plurality of weights ofevidence; and performing defect analysis on one or more groups of theplurality of groups of data having highest ranking. Optionally, thedefect analysis is performed on a parameter level.

In some embodiments, a respective weight of evidence for the respectiveequipment group is calculated according to Equation (1):

$\begin{matrix}{{{woe}_{i} = {{{In}\frac{P\left( y_{i} \right)}{p\left( n_{i} \right)}} = {{In}\frac{\#\mspace{14mu} y_{i}\text{/}\#\mspace{14mu} y_{r}}{\#\mspace{14mu} n_{i}\text{/}\#\mspace{14mu} n_{r}}}}};} & (1)\end{matrix}$

wherein woe, stands for the respective weight of evidence for therespective equipment group; P(yi) stands for a ratio of a number ofpositive samples in the respective equipment group to a number ofpositive samples in all fabrication node groups (e.g., equipmentgroups); P(ni) stands for a ratio of a number of negative samples in therespective equipment group to a number of negative samples in allfabrication node groups (e.g., equipment groups); the positive samplesmeans data including defect information associated with the respectiveequipment group; the negative samples means data in which defectinformation associated with the respective equipment group is absent;#yi stands for the number of positive samples in the respectiveequipment group; #yr stands for the number of positive samples in allfabrication node groups (e.g., equipment groups); #ni stands for thenumber of negative samples in the respective equipment group; #yr standsfor the number of negative samples in all fabrication node groups (e.g.,equipment groups).

In some embodiments, the method further includes processing thefabrication data information to obtain a processed data. Optionally,processing the fabrication data information comprises performing datafusion on biographical data information and defect information to obtaina fused data information.

In one example, processing fabrication data information to obtain aprocessed data includes obtaining raw data information of variousfabrication processes of a display panel, including biographical datainformation, parameter information, and defect information;pre-processing the raw data to remove null data, redundant data, anddummy field, and filtering the data based on pre-set conditions, toobtain a validated data; performing data fusion on the biographical datainformation and the defect information in the validated data to obtain athird fused data information; determining if any piece of defectinformation in the fused data information contains a machine-detecteddefect information and a manually reviewed defect information in a samepiece, and marking the manually reviewed defect information as thedefect information to be analyzed instead of the machine-detected defectinformation, thereby generating a reviewed data; performing data fusionon the review data and the biographical data information to obtain afourth fused data information; and removing non-representative data fromthe fourth fused data information to obtain the processed data. Forexample, data generated in a process in which the glass passes through avery small number of devices can be eliminated. When the number ofdevices the glass passed through is only a small percentage (e.g., 10%)of the total number of devices, the non-representative data will biasthe analysis, affecting accuracy of the analysis.

In one example, the biographical data information (used to be fused withthe review data to obtain the fourth fused data information) includesglass data and hglass data (half glass data, i.e., the history dataafter the complete glass is cut in half). The reviewed data, however, ispanel data. In one example, the glass_id/hglass_id at fab stage is fusedwith the panel_id at EAC2 stage, with redundant data removed. Thepurpose of this step is to ensure the biographical data information atthe fab stage are consistent with the defect information at the EAC2stage. For example, the number of bits in the glass_id/hglass_id is notthe same as the number of bits in the panel_id. In one example, thenumber of bits in the panel_id is processed to be consistent with thenumber of bits in the glass_id/hglass_id. After the data fusion, a datawith complete information is obtained, including glass_id/hglass_id,site information, equipment information, defect information. Optionally,the fused data is subject to an additional operation to remove redundantdata items.

In some embodiments, performing the defect analysis includes performingfeature extraction on parameters of various types to generate parameterfeature information, wherein one or more of a maximum value, a minimumvalue, an average value, and a median value are extracted for at leastone type (e.g., each type) of parameters. Optionally, performing featureextraction includes performing time domain analysis to extractstatistics information comprising one or more of count, mean value,maximum value, minimum value, range, variance, bias, kurtosis, andpercentile. Optionally, performing feature extraction includesperforming frequency domain analysis to convert time domain informationobtained in the time domain analysis into frequency domain informationcomprising one or more of power spectrum, information entropy, andsignal-to-noise ratio.

In one example, the feature extraction is performed on a list ofplurality of groups of data ranked based on the plurality of weights ofevidence. In another example, the feature extraction is performed on oneor more groups of the plurality of groups of data having highestranking. In another example, the feature extraction is performed on thegroup of data having the highest ranking.

In some embodiments, performing the defect analysis further includesperforming data fusion on at least two of parameter feature information,biographical information of the manufacturing process, and defectinformation associated therewith. Optionally, performing data fusionincludes performing data fusion on parameter feature information anddefect information associated therewith. Optionally, performing datafusion includes performing data fusion on parameter feature information,biographical information of the manufacturing process, and defectinformation associated therewith. In another example, performing datafusion on the parameter feature information and biographical informationof the manufacturing process to obtain first fused data information; andperforming data fusion on the first fused data information and defectinformation associated therewith to obtain second fused datainformation, the second fused data information comprising glass serialnumber, manufacturing site information, device information, theparameter feature information, and the defect information. In someembodiments, the data fusion is performed in the general data layer GDL,e.g., by building tables having correlation constructed according touser needs or themes as discussed above.

In some embodiments, the method further includes performing acorrelation analysis. FIG. 11 illustrates a method of defect analysis insome embodiments according to the present disclosure. Referring to FIG.11, the method in some embodiments includes extracting the parameterfeature information and the defect information from the second fuseddata information; performing a correlation analysis on the parameterfeature information and the defect information with respect to at leastone type (e.g., each type) of parameters; generating a plurality ofcorrelation coefficients respectively for a plurality of types ofparameters; and ranking absolute values of the plurality of correlationcoefficients. In one example, the absolute values of the plurality ofcorrelation coefficients are ranked in order from largest to smallest,so that associated parameters that lead to the defect occurrence can beobserved visually. The absolute values are used here because thecorrelation coefficients can be positive or negative values, i.e., therecan be a positive or a negative correlation between the parameter andthe defect. The larger the absolute value is, the stronger thecorrelation.

In some embodiments, the plurality of correlation coefficients are aplurality of pearson correlation coefficients. Optionally, a respectivepearson correlation coefficients is calculated according to Equation(2):

$\begin{matrix}{{\rho_{x,y} = {\frac{{cov}\mspace{14mu}\left( {x,y} \right)}{\sigma_{x}\sigma_{y}} = {\frac{E\left( {\left( {x - \mu_{x}} \right)\left( {y - \mu_{y}} \right)} \right)}{\sigma_{x}\sigma_{y}} = \frac{{E({xy})} - {{E(x)}{E(y)}}}{\sqrt{{E\left( x^{2} \right)} - {E^{2}(x)}}\sqrt{{E\left( y^{2} \right)} - {E^{2}(y)}}}}}};} & (2)\end{matrix}$

wherein x stands for a value for a parameter feature; y stands for avalue for presence or absence of a defect, y is given a value of 1 whenthe defect is present, and y is given a value of 0 when the defect isabsent; μ_(x) stands for a mean value of x; μ_(y) stands for a meanvalue of y; σ_(x)σ_(y) stands for a product of respective standarddeviations of x and y; cov(x,y) stands for a covariance of x,y; andρ(x,y) stands for a respective pearson correlation coefficient.

In another aspect, the present disclosure provides an intelligent defectanalysis method performed by a distributed computing system includingone or more networked computers configured to execute in parallel toperform at least one common task. In some embodiments, the methodincludes executing a data management platform configured to store data,and intelligently extract, transform, or load the data; executing aquery engine connected to the data management platform and configured toobtain the data directly from the data management platform; executing ananalyzer connected to the query engine and configured to perform defectanalysis upon receiving a task request, the analyzer including aplurality of backend servers and a plurality of algorithm servers, theplurality of algorithm servers configured to obtain the data directlyfrom the data management platform; and executing a data visualizationand interaction interface configured to generate the task requests.

In some embodiments, the data management platform comprises an ETLmodule configured to extract, transform, or load data from a pluralityof data sources onto a data mart and a general data layer. The method insome embodiment further includes querying, by a respective one of theplurality of algorithm servers, a first data directly from the datamart, upon receiving an assigned task by the respective one of theplurality of algorithm servers; and transmitting, by the respective oneof the plurality of algorithm servers, a second data directly to thegeneral data layer, upon performing defect analysis.

In some embodiments, the method further includes generating, by the ETLmodule, a dynamically updated table that is automatically updatedperiodically; and storing the dynamically updated table in the generaldata layer.

In some embodiments, the software modules further include a loadbalancer connected to the analyzer. In some embodiments, the methodfurther includes receiving, by the load balancer, task requests andassigning, by the load balancer, the task requests to one or more of theplurality of backend servers to achieve load balance among the pluralityof backend servers, and assigning, by the load balancer, tasks from theplurality of backend servers to one or more of the plurality ofalgorithm servers to achieve load balance among the plurality ofalgorithm servers.

In some embodiments, the method further includes generating, by the datavisualization and interaction interface, a task request; receiving, bythe load balancer, the task request and assigning, by the load balancer,the task request to one or more of the plurality of backend servers toachieve load balance among the plurality of backend servers;transmitting, by the one or more of the plurality of backend servers, aquery task request to the query engine; querying, by the query engine,the dynamically updated table to obtain information on defects of highoccurrence, upon receiving the query task request from the one or moreof the plurality of backend servers by the query engine; transmitting,by the query engine, the information on defects of high occurrence toone or more of the plurality of backend servers; transmitting, by theone or more of the plurality of backend servers, defect analysis tasksto the load balancer for assigning the defect analysis tasks to the oneor more of the plurality of algorithm servers to achieve load balanceamong the plurality of algorithm servers; querying, by the one or moreof the plurality of algorithm servers, the data directly from the datamart to perform defect analysis, upon receiving the defect analysistasks by the one or more of the plurality of algorithm servers; and uponcompletion of the defect analysis, transmitting, by the one or more ofthe plurality of algorithm servers, results of the defect analysis tothe general data layer.

In some embodiments, the method further includes generating anautomatically recurring task request. The automatically recurring taskrequest defining a recurring period for which the defect analysis is tobe performed. Optionally, the method further includes querying, by thequery engine, the dynamically updated table to obtain information ondefects of high occurrence limited to the recurring period; andgenerating, by the one or more of the plurality of backend servers uponreceiving the information on defects of high occurrence during therecurring period, the defect analysis tasks based on the information ondefects of high occurrence during the recurring period. Optionally, themethod further includes receiving input of the recurring period forwhich the defect analysis is to be performed, e.g., by an automatic tasksub-interface of the data visualization and interaction interface.

In some embodiments, the method further includes generating aninteractive task request. Optionally, the method further includesreceiving, by the data visualization and interaction interface, auser-defined analysis criteria; generating, by the data visualizationand interaction interface, the interactive task request based on theuser-defined analysis criteria; transmitting, by the one or more of theplurality of backend servers upon receiving the information on defectsof high occurrence, the information to the data visualization andinteraction interface; displaying, by the data visualization andinteraction interface, the information on defects of high occurrence anda plurality of environmental factors associated with the defects of highoccurrence; receiving, by the data visualization and interactioninterface, a user-defined selection of one or more environmental factorsfrom a plurality of environmental factors; transmitting, by the datavisualization and interaction interface, the user-defined selection tothe one or more of the plurality of backend servers; and generating, bythe one or more of the plurality of backend servers, the defect analysistasks based on the information and the user-defined selection.Optionally, the method further includes receiving input of theuser-defined analysis criteria comprising the user-defined selection ofone or more environmental factors, e.g., by an interactive tasksub-interface of the data visualization and interaction interface.

In some embodiments, the analyzer further includes a cache server and acache. The cache is connected to the plurality of backend servers, thecache server, and the query engine. Optionally, the method furtherincludes storing, by the cache, a portion of results of previouslyperformed defect analysis tasks.

In some embodiments, the data visualization and interaction interfaceincludes a defect visualization sub-interface. Optionally, the methodfurther includes receiving, by the defect visualization sub-interface, auser-defined selection of a defect to be analyzed and generate a callrequest; receiving, by the load balancer, the call request; assigning,by the load balancer, the call request to one or more of the pluralityof backend servers to achieve load balance among the plurality ofbackend servers; transmitting, by the one or more of the plurality ofbackend servers, the call request to the cache server; and determining,by the cache server, whether information on the defect to be analyzed isstored in the cache. Optionally, the method further includes upon adetermination that the information on the defect to be analyzed isstored in the cache, the one or more of the plurality of backend serversis configured to transmit the information on the defect to be analyzedto the defect visualization sub-interface for displaying. Optionally,the method further includes transmitting, by the one or more of theplurality of backend servers, a query task request to the query engine,upon a determination that the information on the defect to be analyzedis not stored in the cache; querying, by the query engine upon receivingthe query task request from the one or more of the plurality of backendservers, the dynamically updated table to obtain information on thedefect to be analyzed; transmitting, by the query engine, theinformation on the defect to be analyzed to the cache; storing theinformation on the defect to be analyzed in the cache; and transmitting,by the one or more of the plurality of backend servers, the informationon the defect to be analyzed to the defect visualization sub-interfacefor displaying. Optionally, the portion of results of previouslyperformed defect analysis tasks includes results of previously performeddefect analysis tasks based on automatically recurring task requests;and results of previously performed defect analysis tasks obtained basedon the query task request.

In another aspect, the present disclosure provides a computer-programproduct, for intelligent defect analysis. The computer-program product,for intelligent defect analysis includes a non-transitory tangiblecomputer-readable medium having computer-readable instructions thereon.In some embodiments, the computer-readable instructions are executableby a processor, in a distributed computing system including one or morenetworked computers configured to execute in parallel to perform atleast one common task, to cause the processor to perform executing adata management platform configured to store data, and intelligentlyextract, transform, or load the data; executing a query engine connectedto the data management platform and configured to obtain the datadirectly from the data management platform; executing an analyzerconnected to the query engine and configured to perform defect analysisupon receiving a task request, the analyzer including a plurality ofbackend servers and a plurality of algorithm servers, the plurality ofalgorithm servers configured to obtain the data directly from the datamanagement platform; and executing a data visualization and interactioninterface configured to generate the task requests.

In some embodiments, the data management platform comprises an ETLmodule configured to extract, transform, or load data from a pluralityof data sources onto a data mart and a general data layer. In someembodiment, the computer-readable instructions are further executable bya processor, in the distributed computing system, to cause the processorto perform querying, by a respective one of the plurality of algorithmservers, a first data directly from the data mart, upon receiving anassigned task by the respective one of the plurality of algorithmservers; and transmitting, by the respective one of the plurality ofalgorithm servers, a second data directly to the general data layer,upon performing defect analysis.

In some embodiments, the computer-readable instructions are furtherexecutable by a processor, in the distributed computing system, to causethe processor to perform generating, by the ETL module, a dynamicallyupdated table that is automatically updated periodically; and storingthe dynamically updated table in the general data layer.

In some embodiments, the software modules further include a loadbalancer connected to the analyzer. In some embodiments, thecomputer-readable instructions are further executable by a processor, inthe distributed computing system, to cause the processor to performreceiving, by the load balancer, task requests and assigning, by theload balancer, the task requests to one or more of the plurality ofbackend servers to achieve load balance among the plurality of backendservers, and assigning, by the load balancer, tasks from the pluralityof backend servers to one or more of the plurality of algorithm serversto achieve load balance among the plurality of algorithm servers.

In some embodiments, the computer-readable instructions are furtherexecutable by a processor, in the distributed computing system, to causethe processor to perform generating, by the data visualization andinteraction interface, a task request; receiving, by the load balancer,the task request and assigning, by the load balancer, the task requestto one or more of the plurality of backend servers to achieve loadbalance among the plurality of backend servers; transmitting, by the oneor more of the plurality of backend servers, a query task request to thequery engine; querying, by the query engine, the dynamically updatedtable to obtain information on defects of high occurrence, uponreceiving the query task request from the one or more of the pluralityof backend servers by the query engine; transmitting, by the queryengine, the information on defects of high occurrence to one or more ofthe plurality of backend servers; transmitting, by the one or more ofthe plurality of backend servers, defect analysis tasks to the loadbalancer for assigning the defect analysis tasks to the one or more ofthe plurality of algorithm servers to achieve load balance among theplurality of algorithm servers; querying, by the one or more of theplurality of algorithm servers, the data directly from the data mart toperform defect analysis, upon receiving the defect analysis tasks by theone or more of the plurality of algorithm servers; and upon completionof the defect analysis, transmitting, by the one or more of theplurality of algorithm servers, results of the defect analysis to thegeneral data layer.

In some embodiments, the computer-readable instructions are furtherexecutable by a processor, in the distributed computing system, to causethe processor to perform generating an automatically recurring taskrequest. The automatically recurring task request defining a recurringperiod for which the defect analysis is to be performed. Optionally, thecomputer-readable instructions are further executable by a processor, inthe distributed computing system, to cause the processor to performquerying, by the query engine, the dynamically updated table to obtaininformation on defects of high occurrence limited to the recurringperiod; and generating, by the one or more of the plurality of backendservers upon receiving the information on defects of high occurrenceduring the recurring period, the defect analysis tasks based on theinformation on defects of high occurrence during the recurring period.Optionally, the computer-readable instructions are further executable bya processor, in the distributed computing system, to cause the processorto perform receiving input of the recurring period for which the defectanalysis is to be performed, e.g., by an automatic task sub-interface ofthe data visualization and interaction interface.

In some embodiments, the computer-readable instructions are furtherexecutable by a processor, in the distributed computing system, to causethe processor to perform generating an interactive task request.Optionally, the computer-readable instructions are further executable bya processor, in the distributed computing system, to cause the processorto perform receiving, by the data visualization and interactioninterface, a user-defined analysis criteria; generating, by the datavisualization and interaction interface, the interactive task requestbased on the user-defined analysis criteria; transmitting, by the one ormore of the plurality of backend servers upon receiving the informationon defects of high occurrence, the information to the data visualizationand interaction interface; displaying, by the data visualization andinteraction interface, the information on defects of high occurrence anda plurality of environmental factors associated with the defects of highoccurrence; receiving, by the data visualization and interactioninterface, a user-defined selection of one or more environmental factorsfrom a plurality of environmental factors; transmitting, by the datavisualization and interaction interface, the user-defined selection tothe one or more of the plurality of backend servers; and generating, bythe one or more of the plurality of backend servers, the defect analysistasks based on the information and the user-defined selection.Optionally, the computer-readable instructions are further executable bya processor, in the distributed computing system, to cause the processorto perform receiving input of the user-defined analysis criteriacomprising the user-defined selection of one or more environmentalfactors, e.g., by an interactive task sub-interface of the datavisualization and interaction interface.

In some embodiments, the analyzer further includes a cache server and acache. The cache is connected to the plurality of backend servers, thecache server, and the query engine. Optionally, the computer-readableinstructions are further executable by a processor, in the distributedcomputing system, to cause the processor to perform storing, by thecache, a portion of results of previously performed defect analysistasks.

In some embodiments, the data visualization and interaction interfaceincludes a defect visualization sub-interface. Optionally, thecomputer-readable instructions are further executable by a processor, inthe distributed computing system, to cause the processor to performreceiving, by the defect visualization sub-interface, a user-definedselection of a defect to be analyzed and generate a call request;receiving, by the load balancer, the call request; assigning, by theload balancer, the call request to one or more of the plurality ofbackend servers to achieve load balance among the plurality of backendservers; transmitting, by the one or more of the plurality of backendservers, the call request to the cache server; and determining, by thecache server, whether information on the defect to be analyzed is storedin the cache. Optionally, the computer-readable instructions are furtherexecutable by a processor, in the distributed computing system, to causethe processor to perform upon a determination that the information onthe defect to be analyzed is stored in the cache, the one or more of theplurality of backend servers is configured to transmit the informationon the defect to be analyzed to the defect visualization sub-interfacefor displaying. Optionally, the computer-readable instructions arefurther executable by a processor, in the distributed computing system,to cause the processor to perform transmitting, by the one or more ofthe plurality of backend servers, a query task request to the queryengine, upon a determination that the information on the defect to beanalyzed is not stored in the cache; querying, by the query engine uponreceiving the query task request from the one or more of the pluralityof backend servers, the dynamically updated table to obtain informationon the defect to be analyzed; transmitting, by the query engine, theinformation on the defect to be analyzed to the cache; storing theinformation on the defect to be analyzed in the cache; and transmitting,by the one or more of the plurality of backend servers, the informationon the defect to be analyzed to the defect visualization sub-interfacefor displaying. Optionally, the portion of results of previouslyperformed defect analysis tasks includes results of previously performeddefect analysis tasks based on automatically recurring task requests;and results of previously performed defect analysis tasks obtained basedon the query task request.

Various illustrative operations described in connection with theconfigurations disclosed herein may be implemented as electronichardware, computer software, or combinations of both. Such operationsmay be implemented or performed with a general purpose processor, adigital signal processor (DSP), an ASIC or ASSP, an FPGA or otherprogrammable logic device, discrete gate or transistor logic, discretehardware components, or any combination thereof designed to produce theconfiguration as disclosed herein. For example, such a configuration maybe implemented at least in part as a hard-wired circuit, as a circuitconfiguration fabricated into an application-specific integratedcircuit, or as a firmware program loaded into non-volatile storage or asoftware program loaded from or into a data storage medium asmachine-readable code, such code being instructions executable by anarray of logic elements such as a general purpose processor or otherdigital signal processing unit. A general purpose processor may be amicroprocessor, but in the alternative, the processor may be anyconventional processor, controller, microcontroller, or state machine. Aprocessor may also be implemented as a combination of computing devices,e.g., a combination of a DSP and a microprocessor, a plurality ofmicroprocessors, one or more microprocessors in conjunction with a DSPcore, or any other such configuration. A software module may reside in anon-transitory storage medium such as RAM (random-access memory), ROM(read-only memory), nonvolatile RAM (NVRAM) such as flash RAM, erasableprogrammable ROM (EPROM), electrically erasable programmable ROM(EEPROM), registers, hard disk, a removable disk, or a CD-ROM; or in anyother form of storage medium known in the art. An illustrative storagemedium is coupled to the processor such the processor can readinformation from, and write information to, the storage medium. In thealternative, the storage medium may be integral to the processor. Theprocessor and the storage medium may reside in an ASIC. The ASIC mayreside in a user terminal. In the alternative, the processor and thestorage medium may reside as discrete components in a user terminal.

The foregoing description of the embodiments of the invention has beenpresented for purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formor to exemplary embodiments disclosed. Accordingly, the foregoingdescription should be regarded as illustrative rather than restrictive.Obviously, many modifications and variations will be apparent topractitioners skilled in this art. The embodiments are chosen anddescribed in order to explain the principles of the invention and itsbest mode practical application, thereby to enable persons skilled inthe art to understand the invention for various embodiments and withvarious modifications as are suited to the particular use orimplementation contemplated. It is intended that the scope of theinvention be defined by the claims appended hereto and their equivalentsin which all terms are meant in their broadest reasonable sense unlessotherwise indicated. Therefore, the term “the invention”, “the presentinvention” or the like does not necessarily limit the claim scope to aspecific embodiment, and the reference to exemplary embodiments of theinvention does not imply a limitation on the invention, and no suchlimitation is to be inferred. The invention is limited only by thespirit and scope of the appended claims. Moreover, these claims mayrefer to use “first”, “second”, etc. following with noun or element.Such terms should be understood as a nomenclature and should not beconstrued as giving the limitation on the number of the elementsmodified by such nomenclature unless specific number has been given. Anyadvantages and benefits described may not apply to all embodiments ofthe invention. It should be appreciated that variations may be made inthe embodiments described by persons skilled in the art withoutdeparting from the scope of the present invention as defined by thefollowing claims. Moreover, no element and component in the presentdisclosure is intended to be dedicated to the public regardless ofwhether the element or component is explicitly recited in the followingclaims.

What is claimed is:
 1. A data management platform for intelligentlymanaging data, comprising: an ETL module configured to extract, cleanse,transform, or load data; a data lake configured to store a first groupof data formed by extracting raw data from a plurality of data sourcesby the ETL module, the first group of data having same contents as theraw data; a data warehouse configured to store a second group of dataformed by cleansing and standardizing on the first group of data by theETL module; a general data layer configured to store a third group ofdata formed by subjecting the second group of data to data fusion by theETL module, wherein the general data layer is a distributed data storagestoring information available for querying; and a data mart configuredto store a fourth group of data formed by transforming the third groupof data by the ETL module, wherein the data mart is a database of NoSQLtype storing information available for computational processing.
 2. Thedata management platform of claim 1, wherein the fourth group of datacomprising a plurality of sub-tables having index relationship in whichdata are categorized based on types and/or rules, the fourth group ofdata available for querying or computational processing.
 3. The datamanagement platform of claim 1, wherein the first group of data storedin the data lake is dynamically updated; and the first group of datacomprises a real-time updated data stored in a Kudu™-based database, ora periodically updated data stored in a Hadoop distributed file system.4. The data management platform of claim 1, wherein at least a portionof the raw data in the plurality of data sources is in a binary largeobject format; at least a portion of the first group of data extractedonto and stored in the data lake is in a compressed hexadecimal format;and at least a portion of the second group of data extracted onto andstored in the data warehouse is obtained by decompressing and processingextracted data.
 5. The data management platform of claim 1, wherein theETL module is further configured to generate a dynamically updated tablethat is automatically updated periodically; and the general data layeris configured to store the dynamically updated table comprisinginformation on defects of interest.
 6. The data management platform ofclaim 1, wherein the data mart is configured to store a plurality ofsub-tables split from a respective one of data tables stored in thegeneral data layer; wherein the plurality of sub-tables comprises: afirst sub-table comprising a plurality of environmental factorscorresponding to user-defined analysis criteria in an interactive tasksub-interface in communication with the data management platform; asecond sub-table comprising product serial numbers; and a thirdsub-table comprising values in the third group of data that correspondto at least one of the product serial numbers.
 7. The data managementplatform of claim 1, wherein the general data layer is configured toperforming data fusion on at least two of parameter feature information,biographical information of a manufacturing process, and defectinformation associated therewith.
 8. An intelligent defect analysissystem, comprising: a distributed computing system comprising one ormore networked computers configured to execute in parallel to perform atleast one common task; and one or more computer readable storage mediumsstoring instructions that, when executed by the distributed computingsystem, cause the distributed computing system to execute softwaremodules; wherein the software modules comprise: a data managementplatform configured to intelligently extract, transform, or load rawdata from a plurality of data sources into a managed data, wherein theraw data and the managed data comprise defect information, and themanaged data is stored in a distributed manner; an analyzer configuredto perform defect analysis upon receiving a task request, the analyzercomprising a plurality of algorithm servers configured to obtain themanaged data from the data management platform and perform algorithmanalysis on the managed data to derive a result data on underlyingreasons for defects; and a data visualization and interaction interfaceconfigured to generate the task requests and display the result data. 9.The intelligent defect analysis system of claim 8, wherein the datamanagement platform comprises an ETL module configured to extract,transform, or load data from the plurality of data sources onto a datamart that is a database of NoSQL type; and upon receiving an assignedtask, a respective one of the plurality of algorithm servers isconfigured to obtain a first data from the data mart.
 10. Theintelligent defect analysis system of claim 9, wherein the ETL module isfurther configured to extract, transform, or load data from theplurality of data sources onto a general data layer that is adistributed data storage storing information; upon performing defectanalysis, the respective one of the plurality of algorithm servers isconfigured to transmit a second data to the general data layer; the ETLmodule is configured to generate a dynamically updated table that isautomatically updated periodically; and the general data layer isconfigured to store the dynamically updated table.
 11. The intelligentdefect analysis system of claim 10, wherein the software modules furthercomprise a query engine connected to the data management platform andconfigured to obtain the managed data from the data management platform.12. The intelligent defect analysis system of claim 11, wherein the datavisualization and interaction interface is configured to generate a taskrequest; the analyzer further comprises a plurality of business servers;upon receiving the task request, the one or more of the plurality ofbusiness servers are configured to transmit a query task request to thequery engine; the query engine, upon receiving the query task requestfrom the one or more of the plurality of business servers, is configuredto query the dynamically updated table to obtain information on defectsof interest, and transmit the information on defects of interest to oneor more of the plurality of business servers; upon receiving defectanalysis tasks, the one or more of the plurality of algorithm serversare configured to obtain the first data from the data mart to performdefect analysis; and upon completion of the defect analysis, the one ormore of the plurality of algorithm servers are configured to transmitresults of the defect analysis to the general data layer.
 13. Theintelligent defect analysis system of claim 12, wherein the task requestis an automatically recurring task request, the automatically recurringtask request defining a recurring period for which the defect analysisis to be performed; the query engine is configured to query thedynamically updated table to obtain the information on defects ofinterest limited to the recurring period; and upon receiving theinformation on defects of interest during the recurring period, the oneor more of the plurality of business servers are configured to generatethe defect analysis tasks based on the information on defects ofinterest during the recurring period.
 14. The intelligent defectanalysis system of claim 12, wherein the task request is an interactivetask request; the data visualization and interaction interface isconfigured to receive a user-defined analysis criteria, and configuredto generate the interactive task request based on the user-definedanalysis criteria; upon receiving the information on defects ofinterest, the one or more of the plurality of business servers areconfigured to transmit the information to the data visualization andinteraction interface; the data visualization and interaction interfaceis configured to display the information on defects of interest and aplurality of environmental factors associated with the defects ofinterest, and configured to receive a user-defined selection of one ormore environmental factors from the plurality of environmental factors,and transmit the user-defined selection to the one or more of theplurality of business servers; and the one or more of the plurality ofbusiness servers are configured to generate the defect analysis tasksbased on the information and the user-defined selection.
 15. Theintelligent defect analysis system of claim 11, wherein the analyzerfurther comprises a plurality of business servers; and the softwaremodules further comprise a load balancer connected to the analyzer, theload balancer configured to receive task requests and configured toassign the task requests to one or more of the plurality of businessservers to achieve load balance among the plurality of business servers,and configured to assign tasks from the plurality of business servers toone or more of the plurality of algorithm servers to achieve loadbalance among the plurality of algorithm servers.
 16. The intelligentdefect analysis system of claim 15, wherein the task requests areassigned to each of the one or more of the plurality of business serversbased on a number of tasks currently performed by each of the pluralityof business servers and an amount of computational burden required byeach task currently performed by each of the plurality of businessservers, to achieve an optimal load balance among the plurality ofbusiness servers.
 17. The intelligent defect analysis system of claim15, wherein the data visualization and interaction interface isconfigured to generate a task request; the load balancer is configuredto receive the task request and configured to assign the task request toone or more of the plurality of business servers to achieve load balanceamong the plurality of business servers; the one or more of theplurality of business servers are configured to transmit a query taskrequest to the query engine; the query engine, upon receiving the querytask request from the one or more of the plurality of business servers,is configured to query the dynamically updated table to obtaininformation on defects of interest, and transmit the information ondefects of interest to one or more of the plurality of business servers;the one or more of the plurality of business servers are configured totransmit defect analysis tasks to the load balancer for assigning thedefect analysis tasks to the one or more of the plurality of algorithmservers to achieve load balance among the plurality of algorithmservers; upon receiving the defect analysis tasks, the one or more ofthe plurality of algorithm servers are configured to query the firstdata from the data mart to perform defect analysis; and upon completionof the defect analysis, the one or more of the plurality of algorithmservers are configured to transmit results of the defect analysis to thegeneral data layer.
 18. The intelligent defect analysis system of claim15, wherein the analyzer further comprises a cache server connected tothe plurality of business servers and the query engine; and the cacheserver is configured to store a portion of results of previouslyperformed defect analysis tasks in a cache.
 19. The intelligent defectanalysis system of claim 18, wherein the data visualization andinteraction interface comprises a defect visualization sub-interface;the defect visualization sub-interface is configured to receive auser-defined selection of a defect to be analyzed and generate a callrequest; the load balancer is configured to receive the call request andconfigured to assign the call request to one or more of the plurality ofbusiness servers to achieve load balance among the plurality of businessservers; the one or more of the plurality of business servers areconfigured to transmit the call request to the cache server; and thecache server is configured to determine whether information on thedefect to be analyzed is stored in the cache.
 20. An intelligent defectanalysis method performed by a distributed computing system comprisingone or more networked computers configured to execute in parallel toperform at least one common task; the method comprising: executing adata management platform configured to intelligently extract, transform,or load raw data from a plurality of data sources into a managed data,wherein the raw data and the managed data comprise defect information,and the managed data is stored in a distributed manner; executing ananalyzer configured to perform defect analysis upon receiving a taskrequest, the analyzer comprising a plurality of algorithm serversconfigured to obtain the managed data from the data management platformand perform algorithm analysis on the managed data to derive a resultdata on underlying reasons for defects; and executing a datavisualization and interaction interface configured to generate the taskrequests and display the result data.